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F  -  6232 
CHAPTER  I 
INTRODUCTION 

This  report,  MATHEMATICAL  report  F-6232,  presents  the  results  of 
research  performed  under  ONR  Contract  N00014- 68-C-0379 ,  and  Mod.  POO]  of 
that  contract,  during  the  period  March  1968  to  September  1969. 

The  objectives  of  the  research  have  been: 

(i)  to  identify  actual  operational  ASW  problems  on  which  game- 
theoretic  methods  give  convincing  evidence  that  valuable  insights 
can  be  provided; 

(ii)  to  formulate  mathematical  problems  which  reflect  the  aspects 
of  actual  ASW  problems  which  are  believed  to  be  critical  for  prac¬ 
tical  operational  solutions; 

(iii)  to  investigate  the  solvability  of  those  mathematical  problems; 

(iv)  to  completely  solve  at  least  a  sample  of  significant  special 
cases  of  the  above  mathematical  problems. 

In  particular,  we  have  given  major  emphasis  to  the  extension  of  research 
performed  by  MATHEMATICA  under  previous  ONR  contracts,  so  that  the  earlier 
models  may  include  more  operationally  significant  factors. 

The  remainder  of  this  report  consists  of  seven  chapters  which  are  sum¬ 
marized  in  the  following . 


Chapter  II  -  A  DESTROYER  VERSUS  SUBMARINE  SEARCH  GAME 


The  basic  problem  considered  in  this  paper  is  the  following: 

An  Evader  E  (enemy  submarine)  is  at  position  Owhen  he  is  "sighted"  by 
a  Pursuer  P  (submarine,  surface  ship,  fixed-wing  aircraft  or  helicopter).  The 
fact  of  the  sighting  is  known  by  F.  Assuming  that  E  has  time  to  take  evasive 
action,  the  problem  to  be  solved  is  the  determination  of  the  path  P  must  follow 
in  order  to  maximize  his  probability  of  delecting  E. 

For  this  problem  we  allow  both  E  and  P  to  have  a  wide  ,  but  simplified 
set  of  strategies.  E  can  choose  a  direction  at  random  and  adopt  a  fixed  speed. 

P  guesses  E's  speed  and  chooses  his  own  speed  and  proceeds  to  Oi  He  must 
select  a  spiral  search  path  which  originates  at  a  point  on  his  trajectory,  this 
point  being  a  function  of  his  assumed  speed  of  E. 

The  strategies  for  P  are  expressable  as  closed,  algebraic  expressions 
which  enables  us  to  compute  them  by  hand.  A  numerical  example  is  computed 
and  show  that  for  nearly  all  strategies  for  E.  The  resultant  strategies  for  P 
are  superior  to  those  obtained  by  inconvenient  and  expensive  computer  techniques. 
It  is  conjectured  that  P's  strategies  are  near  optimal  for  all  of  E's  strategies. 

The  results  and  extensions  of  this  work  include  the  computation  of  the 
probability  of  detection  by  multi-pursuer  forces,  estimating  the  effective  detection 
radius  of  actual  fleet  exercises,  sensitivity  analyses  of  the  problem  parameters, 
strategies  against  multi-evader  forces  ,  and  the  determination  of  the  optimal 
composition  of  mixed  force  ASW  fleets  . 

Chapter  III  -  DISCRETE  TWO-PERSON  GAME  THEORY  WITH  MEDIAN  PAYOFF 
CRITERION 

The  paper  by  John  E.  Walsh  presents  a  basic  definition  of  the  median  pay- 
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off  criterion,  as  it  might  be  applied  to  a  two-person  non-zero  sum  game  (since 
the  payoffs  for  each  player  are  considered  to  be  merely  ordinal,  and  not  strictly 
quantitative  in  the  usual  sense,  the  concept  of  "zero  sum"  should  really  be  re¬ 
placed  by  the  notion  of  "strictly  competitive").  The  median-optimum  strategies 
for  each  player  under  various  circumstances  are  defined,  and  their  existence  is 
shown . 

The  appendix  to  Chapter  III  derives  some  further  results,  which  are  ob¬ 
tained  by  restricting  the  application  of  the  median  payoff  criterion  to  zero-sum 
(i.e.  ,  strictly-compctitive)  two-person  games.  Algorithms  for  finding  the  solu¬ 
tions  under  these  circumstances  are  suggested.  An  outline  of  the  proof  is  also 
sketched1,  showing  that  if  the  payoffs  in  the  two-person  matrix  game  are  not 
determinate,  but  are  each  sampled  from  a  continuous  probability  distribution, 
then  the  median -optimum  payoff  value  will  be  unique  (in  the  main  paper ,  unique¬ 
ness  could  not  be  assured).  This  approach  may  be  valuable  in  cases  where  quanti¬ 
tative  payoff  information  cannot  be  assured,  but  where  preference  orderings  among 
the  payoffs  can  be  reliably  provided. 

Chapter  IV  -  MIXED  STRATEGIES  IN  PRACTICE 

This  expository  paper  develops  an  explanation  of  the  practical  usefulness 
of  the  concept  of  mixed,  or  random,  behavior  in  a  two  person  game  --  in  particular, 
in  a  two-person  zero-sum  game .  It  is  the  thesis  of  this  paper  that  many  of  the 
common  intuitive  objections ,  to  the  use  of  mixed  strategies  in  practice  ,  are  either 
incorrect  or  can  be  avoided  by  suitable  methods  of  choosing  the  mixed  strategies. 
Chapter  V  -  REJECTION  OF  DATA 

This  paper  presents  some  statistical  methods  which  we  believe  will  be  use¬ 
ful  in  situations  where  measurements  are  made  which  must  be  subsequently  as- 
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signed  to  distributions.  There  are  two  real  situations  in  which  problems  of  this 
type  arise,  and  towards  which  the  results  of  this  paper  represent  a  partial  solu¬ 
tion.  The  first,  and  most  important,  is  the  distinction  between  the  probability 
distribution  of  sonar  returns  when  a  real  target  is  present,  and  the  probability 
distribution  of  sonar  returns  when  no  real  target  is  present.  The  second  repre¬ 
sents  the  distinction  between  the  probability  distribution  of  sonar  returns  when 
various  physical  mechanisms  or  transmission  are  operating.  For  example,  de¬ 
tection  ranges  wiien  bottom-bounce  is  present  are  quite  different  from  detection 
ranges  experienced  when  other  phenomena  are  governing. 

Chapter  VI  -  A  SIMPLE  I  -  GAME 

Chapter  VII  -  THE  GENERALIZATION  AND  SOLUTION  OF  THE  SMALL  I- GAME 
DETECTION  MODEL 

These  papers  represent  the  formulation,  and  the  solution,  of  two  families 
or  simplified  games  of  incomplete  information  (I  -  games).  The  I -game  method¬ 
ology  is  the  result  of  an  attempt  to  extend  game  theory  and  to  apply  it  to  situations 
where  the  physical  background,  i.e.  the  rules  of  the  game,  are  not  entirely  known 
to  both  players.  The  examples  treated  here  are  too  simplified  to  be  of  direct  ap¬ 
plicability,  but  are  intended  as  prototypes  for  the  development  of  more  complex 
games  of  incomplete  information  whose  results  might  be  more  directly  applicable. 
Chapter  VIII  -  FORMULATION  OF  A  GAME  OF  SEARCH  AND  "’SUIT 

This  chapter  describes  a  two  stage  game  --  a  searc'  allowed  by  a 

pursuit  game.  Search  games  have  been  treated  for  a  considerable  period  of  time 
by  standard  game-theoretical  methods,  while  games  of  pursuit  have  also  been 
extensively  studied.  This  present  formulation  is  an  attempt  to  develop  a  more 
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realistic  situation,  because  the  outcome  of  the  search  stage  is  determined  by  the 

result  predicted  for  the  pursuit  which  begins  when  the  search  ends.  It  also  pro- 

* 

vides  some  significant  insights  as  to  what  represents  reasonable  behavior  for  a 
Pursuer  and  an  Evader ,  if  the  result  of  the  search  will  not  be  an  immediate  pay¬ 
off,  but  an  ultimate  payoff  determined  by  the  result  of  the  pursuit.  Although  this 
game  has  not  been  solved,  we  present  qualitative  geometric  descriptions  of  the 
expected  solution.  We  believe  that  additional  computation  will  suffice  to  provide 
a  quantitative  specification  of  optimal  behavior  during  the  search  stage  of  such  a 
game . 

In  addition  to  the  authors  of  the  chapters  of  this  report,  valuable  contribu¬ 
tions  have  been  made  by  Louis  Auslander,  Peter  J.  Kalman,  and  Michel  L. 
Balinski.  Some  of  the  contents  of  this  report  were  given  at  informal  briefings  , 
and  we  benefited  from  comments  received.  In  this  connection  we  would  like  to 
thank  Dr.  Paul  L.  Warnshuis ,  Jr.  ,  Naval  Undersea  Research  and  Development 
Center,  San  Diego;  Dr.  Calvin  Sweat,  Naval  Undersea  Research  and  Development 
Center,  Pasedena;  and  Dr.  William  F.  Lucas,  the  RAND  Corporation. 

The  editors  of  this  report  were  Alan  J.  Truelove  and  John  Mayberry. 
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A  DESTROYER  VERSUS  SUBMARINE  SEARCH  GAME 

t 

0 . 0  Introduction  and  Summary  of  Result,1? 

The  basic  problem  considered  in  this  paper  is  the  following: 

AnEvader  E  (enemy  submarine)  is  at  position  0  when  he 
is  "sighted"  by  a  Pursuer  P  (submarine,  surface  ship,  fixed  wing 
aircraft  or  helicopter).  The  fact  of  the  sighting  is  known  by  E. 
Assuming  that  E  has  time  to  take  evasive  action,  the  problem  to 
be  solved  is  the  determination  of  the  path  P  must  follow  in  order 
to  maximize  his  probability  of  detecting  E. 

FOi.  this  problem  we  allow  both  E  and  P  to  have  a  wide , 
but  simplified  set  of  strategies.  E  can  choose  a  direction  at  ran¬ 
dom  and  adopt  a  fixed  sp  :ed.  P  guesses  E's  speed  and  chooses 
his  own  speed  and  proceeds  to  O.  He  must  select  a  spiral  search 
path  which  originates  at  a  point  on  his  trajectory,  this  point  being 
a  function  of  his  assumed  speed  of  E. 

The  strategies  for  P  are  expressable  as  closed,  algebraic 
expressions  which  enables  us  to  compute  them  by  hand.  A  numeri¬ 
cal  example  is  computed  and  show  that  for  nearly  all  strategies  for 
E.  The  resultant  strategies  for  P  are  superior  to  those  obtained 
by  inconvenient  and  expensive  computer  techniques.  It  is  conjec¬ 
tured  that  P's  strategies  are  near  optimal  for  all  of  E's  strat¬ 
egies  . 

The  results  and  extensions  of  thi  work  include  the  computa¬ 
tion  of  the  probability  of  detection  by  multi-pursuer  forces,  estimat¬ 
ing  the  effective  detection  radius  of  actual  fleet  exercises,  sensitivity 
analyses  of  the  problem  parameters,  strategies  against  multi-evader 
forces,  and  the  determination  of  the  optimal  composition  of  mixed 
force  ASW  fleets. 
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0 . 1  Statement  of  the  Problem 

We  consider  the  following  model  in  search  theory  (Fig.  1). 

An  Evader  (E)  is  "sighted"  at  a  point  O,  at  time  t  =  0;  we 
refer  to  this  position-time  combination  as  the  "datum".  Contact  with 
E  is  then  lost,  because  he  takes  evasive  action,  knowing  he  has  been 
sighted . 

.  We  restrict  E's  strategy  as  follows.: 

(i)  E  chooses  a  direction  at  random 
(ii)  E  adopts  a  fixed  speed  v,  0<v<vo. 

A  pursuer  (P)  is  stationed  at  distance  D  from  O  at  time 
t  =  0.  He  proceeds  directly  towards  O  at  maximum  speed,  Q.  P 
makes  a  guess  at  the  speed  v  chosen  by  E.  He  stops  at  the  point 
A  where  he  would  meet  E ,  if  E  had  in  fact  chosen  speed  v,  and 
chosen  the  direction  towards  P;  from  point  A  ,  P  describes  an  out¬ 
ward  spiral  (constant  pitch,  with  center  O) ,  at  his  "search"  speed 
q,  which  will  generally  be  lower  than  his  maximum  speed.  This 
outward  spiralis  such  that  it  matches  E’s  speed  v  exactly,  that 
is,  the  radial  component  of  P's  velocity  is  precisely  v.  Then,  if 
the  guess  v  was  exact,  P  is  certain  to  meet  E  before  he  has  com¬ 
pleted  a  revolution  about  O. 
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Now  suppose  P  has  a  detector,  with  "cookie-cutter"  radius 
R.  Then,  even  if  P  does  not  guess  v  exactly,  he  still  has  a  chance 
of  detecting  E.  .The  question  discussed  in  this  paper  is  ,  "What  'mix' 
of  spirals  should  P  adopt  in  order  to  maximize  Pr(detection  of  E)?" 
We  also  consider  the  "dual"  question,  namely,  "What  mix  of  speeds 
should  E  adopt?" 

E  is  assumed  to  know  P's  distance  at  t  =  0,  but  not  P's 
bearing.  Both  P  and  E  know  each  other's  maximum  speeds. 
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0.2  Summary  of  Results 

The  "mix"  of  spirals  (i.c.  ,  the  corresponding  probability 
density  g*(v),  of  the  choice  of  v)  which  is  optimal  for  P  is  de¬ 
termined  approximately.  A  closed  form  expression  for  g*(v)  is 
found,  the  coefficients  of  which  are  functions  of  the  physical  para¬ 
meters  of  the  model,  and  the  resulting  Pr(detection)  is  calculated. 

The  optimal  strategy  for  E  is  not  found  explii  illy ,  but  v/« 
give  a  heuristic  argument  showing  that  E's  strategy  is  almost 
identical  with  the  strategy  obtained  for  P. 

Since  this  general  solution  is  only  approximate  ,  an  exact 
solution  for  a  particular  case  was  computed,  using  linear  program¬ 
ming.  It  was  necessary  to  restrict  P's  strategy  to  a  mix  of  four 
particular  strategies.  The  results  confirm  that  the  approximate 
solution  is  very  nearly  as  good  as  this  (restricted) exact  solution. 

The  strategy  for  P,  g*(v),  and  the  resulting  Pr(detection) , 
are  graphed  below.  [Figs.  2  and  3],  E's  maximum  speed  is  taken 
as  1 ,  so  that  P's  speed  is  expressed  in  terms  of  this  unit. 
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0 . 3  Comparison  of  Results  with  Danskin's  Helicopter  Search 

Danskin  (1),  considered  a  Helicopter  (P)  versus  Submarine 
(E)  game ,  and  introduced  the  'speed  circle'  concept  (see  following 
section). 

The  differences  between  his  model  and  ours  are: 

(a)  The  helicopter  performs  a  'dip'  for  a  finite  specified 
period  of  time,  with  a  cookie-cutter  detector. 

(b)  He  assumes  the  helicopter  takes  zero  time  to  fly  from  one 
dip-point  to  any  other  dip-point  in  the  search. 

Danskin's  solution  specifies  identical  strategies  for  the  alloca¬ 
tion  of  Helicopter  search  effort,  and  the  allocation  of  Submarine 
choice  of  v  ,  namely  the  g^(v)  function  shown  below. 


FIGURE  4  -  Danskin's  solution  for  both  P  and  E  ..  gp(v)  . 

In  our  case  ,  we  agree  (heuristically)  that  the  strategies  for  P 
and  E  should  be  approximately  identical,  under  reasonable  assump¬ 
tions  on  cookie -cutter  range,  initial  distance  D  ,  and  relative  speeds 
(see  section  10,  'A  Smeared-Diagonal  Zero-Sum  Game'). 
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However,  in  our  problem  P  is  constrained  to  move  in  a 
continuous  path.  There  are  two  effects  which  tend  to  counteract 
each  other.  (In  the  description,  we  make  use  of  the  speed  space 
transformation,  described  in  the  following  section). 

(a)  P  arrives  at  the  rim  of  the  speed  circle  when 
t,  the  elapsed  time,  is  still  small.  His  relative 
cookie-cutter  radius  is  large,  so  search  circles 
near  the  rim  are  more  effective  than  in  the  heli¬ 
copter  case. 

(b)  By  the  time  P  gets  to  (or  near)  O,  hiscookie- 
cutter  is  small.  On  the  other  hand,  he  has  more  ex¬ 
cess  speed  to  use  on  transverse  motion,  since  his 
required  radial  speed  is  small  or  zero,  to  match 
that  assumed  for  E.  Hence,  he  covers  more 
ground. 

Our  recommended  strategy  g**(v),  for  the  numerical  case 
considered,  does  not  differ  too  much  from  Danskin’s  solution  gjp(v). 
In  fact  it  shows  a  slight  "hump"  for  the  middia  range  of  v.  (There 
is  an  anomalous  spike  at  v  =  1 ,  the  reason  for  which,  explained 
later  is  to  deal  with  the  difficulty  of  covering  values  of  v  near  the 
rim  v  =  1;  this  anomaly  was  not  considered  by  Danskin,  who  as¬ 
sumed  that  the  successive  helicopter  circles,  having  decreasing 
radius  in  speed  space ,  could  be  packed  together  with  sufficient  ac¬ 
curacy.  ) 
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0.4  Uses  of  the  Results  and  Indicated  Further  Work 

The  folk  ing  are  possible  uses  of  our  single-speed  search 
model  in  realistic  situations: 

(a)  Computation  of  Pr (detection)  by  Multi -Unit 
Forces 

In  specified  situations,  e.g.  ,  4  destroyers 
situated  10  miles  away  from  the  point  of  detection, 
it  is  possible  to  calculate  the  probability  of  detec¬ 
tion,  or,  conversely,  the  probability  that  the  sub¬ 
marine  will  escape. 

(b)  Estimation  of  Effective  Detection  Radius 

Suppose  that  destroyer  vs.  submarine  (or 
similar)  games  are  observed,  in  situations  which 
are  reasonably  close  to  those  assumed  in  the  mod¬ 
el. 

Then  we  may  observe  (over  repeated  trials) 
the  probability  of  successful  search  and,  in  fact, 
the  time  and  position  of  the  final  detection.  From 
these  data,  we  can  then  estimate  certain  para¬ 
meters  of  the  model  ,  e.g.  effective  "cookie -cutter 
detection  radius . 

(c )  Optimization  of  Equipment  Parameter 

The  parameters  of  the  problem  could  be 
varied  to  yield  answers  to  the  following: 


II  -  13 


‘I 


(i)  How  sensitive  is  Pr (detection)  to  the  ratio 

* 

of  maximum  speeds ,  P  to  E  ? 

(ii)  What  is  the  effect  of  replacing  the  assumption 
that  E  knows  P's  distance  D  ,  by  the  assumption 
that  E  has  an  a  priori  distribution  on  D  ? 

(iii)  Same  as  (ii),  if  E  has  only  an  a  priori  distri- 
bution  on  P's  maximum  speed  Q  ,  and/or  on  P's 
search  speed,  q  . 

(d)  Mixed  Force  Problems 

Suppose  that  the  Pursuer's  force  consists  of  two  or 
more  of  the  following : 

Surface  ships , 

Pursuit  submarines, 

Helicopters  (e.g. ,  Aircraft  carrier  based), 

Fixed-wing  aircraft. 

Using  suitable  mathematical  programming  techniques, 
we  can  determine  the  optimal  allocation  of  portions  of  the  search 
areas  to  the  various  force  components . 

We  can  rilso ,  for  the  various  components ,  determine 
the  utility  of  different  initial  distances,  and  approach  speeds. 

(e)  Optimal  Composition  of  Mixed  Force  ASW  Fleets 

For  ASW  fleets ,  it  is  better  to  have  a  few  large  mixed 
forces,  or  more,  but  smaller  groups?  (Assuming  that  initial 
contacts  with  Evaders  are  made  randomly  over  the  tactical 
area).  Graphs  showing  Pr  (detection)  vs.  Force  size,  with 
initial  distance  from  the  datum  as  a  parameter ,  would  be  a 
start  in  solving  this  problem. 
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0 . 5  Remark  on  More  Sophisticated  Strategies 

The  Pursuer  has  been  restricted  to  outward  spirals,  starting 
from  some  point  on  the  line  between  his  initial  position  and  the  datum 
while  the  Evader  has  been  restricted  to  single  direction,  single  speed 
strategies  (following  Danskin).  The  question  naturally  arises,  whether 
significant  improvement  on  either  side  could  be  obtained  by  using  more 
sophisticated  straegies. 

First,  since  E  knows  that  some  interval  will  elapse  before 
P-  can  come  within  range ,  there  is  an  argument  for  proceeding  at 
maximum  or  high  speed  for  most  of  this  interval.  A  two-stage  speed 
model  is  considered  later  in  the  paper,  but  no  results  have  been  ob¬ 
tained  . 

There  seems  no  good  reason  for  E  to  choose  other  than  a 
fixed  direction,  assuming  that  he  cannot  detect  P's  search  path. 
Zig-zagging,  back-tracking,  etc.  ,  would  mean  that  less  potential 
ground  is  covered,  from  P's  point  of  view,  at  a  greater  speed. 

Next  we  come  to  P's  strategy.  It  is  seen  that  the  recom¬ 
mended  mix  of  spirals  results  approximately  in  a  "uniform"  coverage 
of  the  speed  circle,  which  is  precisely  the  solution  that  Danskin  ob¬ 
tained  in  the  simpler  game.  Wherever  E  places  himself  in  the  speed 
circle,  the  probability  of  detection  is  the  same.  It  is,  heuristically , 
clear  (as  mentioned  later  on  in  section  10)  that  any  optimal  strategy 
not  restricted  to  spirals  for  P  must  result  in  uniform  coverage.  It 
appears  likely  that  our  method  is  at  least  as  economical  as  any  other, 
i.e.  that  any  other  mix  of  strategies  would  result  in  a  value  of  the 


game  (Pr(detection))  equal  to  or  lower  than  the  value  we  ob¬ 
tained  here  .  If  this  is  true ,  then  our  class  of  strategics  is 
’’complete"  in  the  usual  statistical  sense  . 
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1  .  Preliminaries 


1.0  The  Model 

At  time  t=0  ,  an  Evader  (E)  is  sighted  at  a  point  0.  The 
position/time  combination  being  referred  to  as  the  'datum’.  E 
realizes  he  has  been  sighted,  dives,  and  adopts  the  following  strategy: 

(i)  E  chooses  a  direction  at  random,  ' 

(ii)  E  chooses  a  fixed  speed  v,  0<v<v^  . 

A  Pursuer  P  is  located  at  distance  D  from  O  ;  E  does 

not  know  E's  speed  or  bearing.  P  knows  E's  maximum  speed 

v  ;  E  knows  P's  distance  D  ,  maximum  and  search  speeds  (see 
o 

below)  Q,  q  ;  and  each  is  aware  that  the  other  has  this  information. 

P  proceeds  at  maximum  speed  Q  towards  the  datum  point, 
and  adopts ,  from  some  point  onwards ,  with  speed  q  <  Q  ,  a  strategy 
chosen  from  a  set  to  be  defined. 

P  and  E  are  blind  to  each  other ,  except  that  if  E  comes 
within  range  R  of  P  (the  'cookie -cutter'  assumption),  P  is  con¬ 
sidered  to  detect  E  . 

1 . 1  The  'Speed  Circle1  Transformation 

It  is  convenient  to  introduce  the  following  mathematical  trans¬ 
formation  .  O  is  kept  fixed  . 

The  bearings  of  P  and  E  from  O  are  left  unchanged .  But 
radial  distances  from  O  are  multiplied  by  the  factor  1/t  ,  where  t 
is  the  time  elapsed  since  the  datum.  It  will  be  seen  tha£  the  position 
of  E  in  this  transformed  picture  (called  'speed  space')  will  be  a 
fixed  point,  somewhere  in  the  circle  center  O  ,  radius  vq  ,  called 
the  speed  circle,  and  this  point  will  be  labelled  'v'  . 
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Since  P’s  radial  component  of  velocity  is  v  ,  his  path  will 
he  represented  by  a  circle  radius  v.  In  fact,  if  P  continues  his 
spiral  (in  real  space)  indefinitely,  he  will  trace  this  circle  over  and 
over . 


Speed  Space 


FIGURE  5  -  Real  and  Speed  Space 

2  .  Pursuer's  Speeds  and  Strategy  Class  S 

Generally  P  will  be  assumed  to  have  2  available  speeds: 
The  maximum  speed  of  Q  at  which  the  detection  probability 
is  so  small  that  it  can  be  ignored  and  a  search  speed  of  q ,  at  which 
the  cookie-cutter  assumption  holds. 


Next,  P  adopts  (in  real  space)  a  spiral,  contained  in  the 
class  S  ,  corresponding  in  speed  space  to  a  circle  and  steaming  at 
•search  speed*  q  . 


3  ,  Area  of  Coverage. 

In  speed  space,  the  radius  of  detection  is  R/t. 

The  area  covered  by  P's  search  may  be  shown  thus: 


If  the  circle  starts  from  point  v  ,  we  will  denote  this  shaded  area 
by  a 

v 

Consider  the  fixed  circle,  in  speed  space,  radius  v'  . 

How  much  of  the  perimeter  of  this  circle  will  we  cover  by 
a^  ?  Jf ,  in  fact  a^  still  has  significant  width  after  a  rotation  of 
2  1? ,  we  suppose  that  the  shaded  area  is,  from  then  on,  displaced 
radially  just  enough  to  avoid  overlap. 


This  means  that  a  coverage  of  the  perimeter  of  more  than 
2  it  v  will  have  meaning  ,  as  will  become  clear  . 

In  speed  space,  the  speed  along  the  arc  (radius  v  )  is 


This  is  because  in  real  space,  P  must  steam  radially  away  from  O 

with  component  speed  v  to  'stay  where  it  is'  (like  Alice)  in  speed 

2  2  1/2 

space.  This  leaves  a  component  (q  -v  )  transversely  which  in 

2  2  1/2 

speed  space  becomes  (q  -  v  )  /t  . 
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long  as 


The  shaded  area 


av  ,  will  cover  part  of  the  arc  at 


so 


(3.2) 


> 


i.e . ,  up  to  the  time 


(3.3)  fcj  =  R/  }v  -  v’|  . 


Distance  travelled  along  the  arc  at  v  at  time  t^  is 


DA 


*1  2  2  1/2 

=  f  —  dt  s  (q2 .  v2)  log  (tj/tv) 


1/2 


t=t 


2  2 


1/2 


=  (q  -  v  )  log 


R/  [v  -  y'l 
D/(Q  +v> 


using  (2.2),  (3.3). 

How  much  of  the  arc  at  v’  is  'covered’? 


Obviously,  (DA ) .  (-^r)  ,  whethe  r  v' >  v  or  <v  . 
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Let 


D 

0 

0 

0 

D 

tJ 

D 

D 

D 

0 

0 

D 

D 

D 

0 

u 

!i 

I 
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4  .  Probability  density  function,  g»(v),  for  mix  of  P!s  strategies  . 

Let  P  choose  strategy  (circle  in  speed-space  radius  v)  with 
the  following  p.d.f. 

v 

v  r° 

(4*1)  g  (v) ,  0  v  <  where  J  g  (v)  =  1  . 

v  =  0 


Given  v'  ,  0  <  v*  <  v^  what  is  the  expected  'coverage*  (in  the  sense  of  the 
previous  section)  of  the  arc  with  radius  v'  ?  It  is 


(4.2) 


v=v'+u 

K(v’)=f 

V=V*-u 


2.1/2 

v  ) 


R/|v-v'| 

108  t  W(q+v)  3 


* 

g  (v)  dv  . 


What  is 


H(v’)  =  Pr  (P  kills  E  {  P*s  stratf  gy  =  g  (v)  , 


E's  strategy  =  v’)  ? 


It  is 


H(v')  =  min  fK(v')  /  21 Tv',  l}  . 

We  make  the  assumption  that  it  is  sufficiently  accurate  to  take 

K(v')/27Tv'  <l,v'f[0,v]  . 

o 


11-25 


Then,  referring  to  "A  Smeared-Diagonal  Zero-Sum  Game", 
Section  10  of  this  paper  we  see  that  P's  optimal  strategy  will 
yield 


H(v')  =  constant,  v1  c  [0,  v 

O' 


i.e . , 


K(v')/v'  =  constant  . 

We  seek  the  g*(v)  that  satisfies  this  constraint.  It  is  heu- 
ristically  apparent  that  E's  optimal  strategy  will  also  be  to  choose 
v!  using  the  same  distribution  g*{v)  .  [See  note  on  a  smeared 
diagonal  zero-sum  game  below]  . 


Without  loss  of  generality,  take  v  s  1  from  now  on.  We  write  g  (v)  in  the 
following  form: 


. .  o \  *  .  f  .  2  3  4,  .  2  2r  1/2 

(4.3)  g  (v)  =  iaQ  v  +  aj  v  +  &z  v  +  a3  v  j  (q  -  v  ) 

,  .  .2  2.-  1/2 
=  g(v)  .  v  .  (q  -  v  ) 

This  imposes  some  restriction  on  g(v),  of  course,  but  this  form  will  be 
sufficiently  general  for  practical  purposes  . 

Note  that ,  since 


I 


we  should  have 


(4 .4)  a  I  +  a,  I,  +  a  I  +  a_  1.  =  1 

01122334 


where 


2,-  1/2 

v  ) 


dv  . 
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Set  a 


Then  : 
Then, 


(4.5) 


Using  a  certain  criterion,  we  shall  calculate  the  ratios 


a  : 
o 


a 


3  ’ 


=  -  1  arbitrarily,  then  compute 


a  I,  +  . .  .  a,  I.  =  C  . 
o  1  3  4 


replace  a  ,  a.  ,  a_,  a,  by  a  /C  ,  a,/C  ,  a_/C  ,  a,/C  . 
O  l  c  5  o  1  c  5 

(4.4)  will  be  satisfied. 


Returning  to  (4.2)  we  have 


y  =  v'  4  u 


K(v') 


- 


log 


~v  =  v' 


u 


R/|v-v'| 

f"6T(ai7) }  g(v)  ir 
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a(Q 


CO 

+Vj  if 

y  =  i 


- u  (dy )  + 


(-  <Iz)  ] 


using  (3.8),  (3.10),  (3.13),  and  (3.11) 


a(Q 


r  r 

ij  ---u2.dy  +y  ---U2.dz] 


y  n  oo 

l 


a(Q 


+  *'>  [J 


l°g  y  g(v'  “  u)  .  u  .  dy  [case  (  i)  ] 


y  s  oo 

l 


Now,  in  case  (ii),  since 


+  f  log  z  g(v’  +  u)  ’ .  u  dz]  [case  (ii)] 

*4  =  » 


2  3 

g(v)  =  a  +a,  v+a-v  +a,v 
o  1  c  5 


2  3 

we  have  g(v*  +  u)=  a  +  a  (v1  +  u)  +  a,  (v'  +  u)  +  a  (v*  +  u) 

O  1  « 


(drop  the  (')) 


=  (ao+al  v+a2  v  +a3v  ) 


+  u  (aj  +  2  aj,  v  +  3  a^  v  ) 


+  u  (a2  +  3  a3  v) 


+  u  (a3) 


=  F 


+  u  .  F, 


+  u2  .  F„ 


+  u  F3  . 


Let 


F(u)  =  F  +  u  F,  +  u2  F.  +  u3  F_ 
o  1  2  3 


For  case  (i),  we  get  F(-u) 
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Now  for  m  >  2 ,  a  >  - 1  , 


(4.6) 


I(m)  = 


log  X 

(x  +  a) 


du  =  [  - 


log  x 


1 


{ 


1 


(m-1 )( x+a)' 


1 


,m-l 


1 


(m-l)a  c  .  ...  .m-2  ,m-3  ‘  m-4,  ,  .2 

(m-2)(x+a)  (m-3)  a tx  +  a)  2a  (x  +  a) 


a  (x+a)  (m-1  )a 


*  f—1-  c  — 

(m-l)a  (m-2)(l  +  a) 


1 


m-2 


+  . .  . 


m-3  m-2 

a  (1  +  a)  a 


log  (1  +  a)  ) 


and,  for  a  «  1 ,  we  get,  using  , 


log  (1  +  a)  =  a  -  +  ^ 


,  2  3  4 

(4.7)  1(2)  =  -±log  (1  +a)=  -  (  1  -|  +  ~  +  “-) 


I(3)  =  7"[Tr;-rloe(I+a)J 

2a 


J_  ,  1  _  ,  2  _2  3.3 K  _  l,  1  _  .  1  2  3  3. 

a 


T2a("2a+3  "4a)=a("4a+3a  "8a) 


I(4,""3la[2(l’a)2+a<ll+a>'al2l°8  <1+al] 

=  -i<-^2+ja3) 
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1(5)  = 

« 

let 

Thus  ,  in  (ii) 

(4.8) 

(writing  now 

(4.9) 

to  obtain 

(4.10) 


1 


—  [ - - -  + - 1 -  + - 

4a  t  \  2  2 

3(1  +  a)  2a(l+a)  a  (1  +  a) 

JL/  13. 

3  (  "  16  a  ) 


I  (2)  -•  1(2)  with  a  replaced  by  -a 


-  log  (  1  +  a )  ] 

a 


,  I  (3)  =  ---  etc . 


/ 


log  z  .g(v  +  u)  u 


z~m 


dz 


v  for  v'  )  make  the  substitution 


J  log  z  .  (Fq  +  Fj  .  u  +  F^  .  u^  +F^  uV^ 


dz 


z  ■•-<*> 

1 


./  log  2  {  F  [  -a-(g  f  +  F.  [a(f  +-v,-}  ]3  +  ...  }  d 
J  6  o  z  -  a  l  z-aJ 


=  F  .  l"(2)  .  a2(Q  +v)Z  +F.  I  "  (3)a3(Q  +v)3 
o  l 

+  F2  l"(4)  a4  (Q  +  v)4  +  F3  l"(5)  a5  (Q  +  v)5 
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Similar  W 


1 

(4.11)  /  log  t  Fo  [  -F1...}dy  =  Fol(2)a2(Q+v)2 

y =00 

-  Fj  1(3)  a3  (Q  +  v)3 

'  +F2  1(4)  a4 (Q  +v)4 

-  F3  I(5)a5(Q  +y)5  . 

Hence ,  from  (4.5), 

,4'12)  kw=T(qTT)[  /-  +/"  1 

=  -  I(Q  V  V j  [Fo  .  aZ(Q+v)2  .  {  I(2)  +  I“(2)] 

+  FZ  .  a3(Q  +v)3  .  [  -  1(3)  4  I"(3)  } 

+  F2  .  a4(Q  +v)4  .  {  1(4)  +I-(4)} 

+  F3  .  a5(Q  +v)5  .  {  -  1(5)  +jf(5)  }  ] 

*  '  ItoTTj '  aZ(Q  +v)2  fF0  [ "  2  '  l“2  ‘  !a4] 

+  F1  (Q+v)(^  +  |a3] 

+  F2  (Q  +v)2  [  -  |a2] 

+  F3  (Q  +v)3[  |  a3] 


Therefore 


(4.13) 


Where 

(4.14) 


1  O  O  O  A 

-  .  K(v)  =  (Q  +  v)  {  F  (2  +  -  a  ^  +  -  a  ) 
av  0  3  5^ 

+  (Q+v)  F  (  -  l/2a  -  3/4  a") 

+  (Q  +  v)2  F2  (|a2)  +  (Q  +v)3  F3  (  -  |  a3)  } 

2  3 

=  P  .  (Q  +  v)  (aQ  +  aj  v  +  n2  v  +  a3  v  ) 

N 

2  2 
+  S  .  (Q  +  v)  (a2  +  2  a2  v  +  3  a3  v  ) 

+  T  .  (Q  +  v)3  (a2  +  3  a3  v) 

+  U  .  (Q  +  v)4  (a3) 

P=(2  +|a2+|a4),  S  =  (-*2-a-|a3),  T=|a2,  U  =  -^a3 

=  {  aQ  .  Q  .  P  +  aj  .  Q2  .  S  +  a2  .  Q3  T  +  a3  Q4  U  } 

+  v  {a  P  4  a,  Q  (P  +  2  S)  +  a,  Q2  (2  S  +  3  T)  +  a,Q3(3T  +  4U)} 

+  v2  {a  (P  +  S)  +a2Q(P  +  2S  +  3T)  +a3Q2(3S  +  9T  +  6U)} 

+  v3  (a  (P  +  2S  +  T)  +  a3Q(P  +  6S  +  9T  +  4U)} 

+  v4  [a,(3T  +  U)} 


We  wish  —  K(v)  to  be  approximately  constant.  So  we  take 

cL 

V 

(4.15)  a3  =  -  i  . 

_  (P  +  6S  +  9T  +  4U) 
a2  =  -  a3  .  Q  .  p  +  2S  +  T 

-  a2  .  Q  (P  +  2S  +  3T)  +  a3  .  Q  .  (3S  +  9T  +  6U) 
ai  "  P  +  S 

a  (Q  +  2S)  +  a2  .  Q2  (2S  +  3T)  +  a3  Q3  (3T  +  4U) 


(The  choice  of  a.  -  -  1  is  arbitrary). 
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6.  Numerical  example. 

Take  Q  =  2. 0 

q  =2.0 
R  =  .05 
D  =  2.0 

The  'recommended1  values  for  a^,  a^,  a^,  a^  using  (4. 15) 

are  as  follows  (with  taken  arbitrarily  as  -1.  0) 

(6.1)  aQ  =  7.796 

e  -3.  901 
a2  =  1.902 
a3  =  -1.  000 

On  substituting  q  =  2.  0  in  equations  (5.1)  -  (5.4),  we  obtain 


Ij  =  .934 
1,  =  .614 

13  =  .456 

14  =  . 362 

Then  C  =  a^I^  +  .  .  .  a3I4  =  5.  392,  C  *  =  0.1854. 
Therefore,  the  recommended  strategy  for  P  is  given  by 


g*{v)  =  C"1  (a 


0 


v  +  a^ 


a2V 


+  a 


4.  .  2 

iv  )  (q  - 


2, 

V  ) 


-1/2 


We  tabulate  g*(v)  at  intervals  of  v  =  .  1,  in  table  1. 

(The  columns  headed  g‘**(v),  g$j  p(v)  are  discussed  later.) 
As  a  check,  note  that  if  the  numerical  integration  formula 


is  used,  we  obtain  j  g*(v)  dv  =  0.  990  approximately. 
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(1) 


(2) 

g*(v) 


(3) 

g**(v) 


V 


0.0 

0.000 

0.1 

0.275 

0.2 

0.449 

0.  3 

0.744 

0.4 

0.941 

0.5 

1.  112 

0.6 

1.257 

0.7 

1.374 

0.8 

1.462 

0.  9 

1.518 

0.95 

(1.529) 

1.0 

1.541 

(3) 

g**(v) 

(4) 

f’lpW 

.000 

0.000 

.262 

0.  261 

.4? 

0.480 

.7 

r  wgptli 

•  ^ 

*  < 

1.0. 

W  KSS 

1.195  • 

1 

V  0.965 

1.311 

1.  153 

1. 382 

1 . 485 

1.435 

2.036 

(1 . 93 1) 

•  (2.459) 

2.914 

2.882 

Table  I  -  Recommended  Strategies  for  the  Pursuer,  P, 

(Note:  For  v  =  0.  95,  interpolated  values  have  been  calculated.) 

(Note:  For  v'  =  1,  u  =  ■  — (Q  +v')  =  — — — - —  (2.  0  +  1.  0)  =  . 

D  +  R  2.0  +  0.05 


~  J^CT.  - - - 


I 

I 

I 

I 

I 

w 

* 

t 

I 

1 


0 

li 


Then  the  following  ’Kill  probabilities’  are  obtained 
(Col.  3  is  discussed  below) 


(1) 

If  E  adopts  speed  v’ 
with  random 
azimuth 

(2) 

2tt  x  P's  Kill 
Probability,  using 
strategy  g* (v) 

(3) 

27T  x  P’s  Kill 
Probability,  using 
strategy  g*  (v) 

.025 

.232 

.221 

.05 

.268 

.755 

.075 

.284 

.271 

•1 

.289 

.274 

.2 

.289 

.265 

.3 

.250 

•4 

.288 

.233 

1  .5 

.287 

.221 

:  -e 

.286 

.221 

!  .7 

.283 

.2TT 

i  .8 

.279 

.289 

!  .9 

.265 

.358 

j  1.0 

* 

.133 

.221 

Table  2  -  Kill  Probabilities 


Thus,  when  strategy  g* (v)  is  used,  the  approximate  relation 
P(Kill  E  uses  strategy  v'  )  =  constant 


11 

11 

u 


is  demonstrated,  except  for  v*  in  the  region  of  1.0,  which  is 
discussed  below.  (Note:  In  the  numerical  integration  used 

for  this  table,  an  interval  of  0.001  was  employed). 

The  kill  probability  for  the  g**(v)  case  was  not  calculated  explicitly 
(see  section  7  below),  but  a  rough  calculation  given  below  shows  a 
typical  kill  probability  of  0.  267. 


■»*  Vi 


7 .  Correction  to  g*(v)  at  the  rim  of  the  Speed  Circle 

The  approximation  used  in  Section  4,  assumes  that  g*(v)  is 
defined  in  (0,®).  In  fact,  g*(v)  is  truncated  at  v  =  vq  (=1 ,  by  choice  of 
units ) . 

Consider,  for  E,  the  maximum  speed  strategy  v'  =  v  .  Then, 
in  eq.  (4.5)  we  assumed 

v  =  v'  +  u  , 

(7.1) .  K(v')  =  /  H(|v-v'|)v  g*(v)  dv 

v  v  =  v*  -  u 

where  K(v')  was  the  "coverage"  (in  the  sense  of  Section  3,  of  the  arc  v1 , 
and  H(|v  -  v’  |)  is  a  function  of  v  which  is  approximately  symmetric  about 
v  -  v' . 

In  reality,  we  have 

1  °  -1 

(7.2)  — ,  K(v')  =  /  H(|v-v'|)v  g*(v)  dv 

v  =  v*  -  u 

Assume  that  v  *  g*(v)  is  approximately  constant  within  the 
above  limits  of  integration ,  then 

0  v?  =  v'  +  u 

(7.3)  '  ~  (1/2)  J 

v’  =  v'  -  u  v'  -  v'  -  u 

We  chose  g*(v)  so  that  K(v')/vr  would  be  appi  oximately 
constant,  but  from  (7.3)  it  is  clear  that  K(v')/v'  will  drop  to  one  half 
of  this  constant  value  ,  for  v'  in  the  neighborhood  of  v'  =  1  . 
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Now  u  =  (Q+v1);  let  g*(l)  =  fj  ,c  =  fj/(l  +u  fj/2)  and 

note  that  f^(l  -  .uc/2)  =  c* 

We  attempt  to  correct  the  error  referred  to  above,  by  adding 
a  "wedge"  of  probability  mass  to  g*(v)  in  the  range  (1  -  u,  1).  Let  us, 
therefore,  change  g*(v)  to  g**(v),  where 


r  g*(v).  (1  -  u  f/2)  0  <:  v  <_  1  -  u 

g**(v)  =  v'  -  (1  -  u) 

g*(v)  (1  -  u  c/2)  +  e -  »  1  -  u  <  v  <  1 


u 


Note  that 


Also 


1  1 

I  g**  (v)  dv  =  (1  -  uc/2)  /  g*(v)  dv  +  uc/2  =  1 . 

0  0 


g**(l)  *=f  (1  -uc/2)  +c=  2e 


Relatively  speaking,  our  transformed  g**(v)  has  doubled  the  weight  assigned 
to  v'  "  1 ,  and  this  extra  weight  tapers  off  (linearly)to  zero  at  v!  =  1  -  u. 

For  the  numerical  example  ,  we  give  the  tran  sformed  in 

Table  I,  col.  (3). 
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It  is  seen  that  g**(v)  is  quite  close  to  the  strategy  g|p(v), 
discussed  below.  Since  kill  probabilities  are  calculated  for  g^p(v),  they 
can  be  expected  to  be  very  similar  for  g**(v),  and  hence  were  not 
calculated  for  this  latter  case.,.  explicitly  .  However,  a  rough  idea 
is  obtained  by  applying  the  correcting  factor,  0.  9454  in  this 
case,  to  a  'typical1  kill  probability  in  the  middle  range  for  v 
in  column  (2)  of  Table  2,  say  the  0.  283  value.  We  obtain  0.  267; 
this  compares  very  favorably  with  the  minimum  value  of  0.  221 
obtained  with  the  LP  solution,  restricted  to  a  mixture  of  four  pure 
strategies.  Of  course,  if  more  than  four  strategies  were  mixed,  the 
LP  value  would  rise.  Also,  we  have  not  checked  that  our  'ad-hoc' 
solution  surpasses  the  0.  267  at  the  two  end-points,  v  =  0,  1. 
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Solution  of  the  Game  by  Linear  Programming  Methods. 

Suppose  we  write  P's  strategy  in  the  form  given  in 
(4.  3),  namely 


g*(v)  =  g(v).  v.  (q2  -  v2) 


-1/2 


n 


and,  now,  let 

g(v)  =  a  +  a,  v  +  a,  v2  +  ...  +a  v“. 
6V  1  o  1  2  n 

We  may  consider  each  term, 


a.v 

l 


as  corresponding  to  a  'pure  strategy';  that  is,  given  the  p.  d.  f. 
g(v)  -  a.v1 

(suitably  normalized),  P  samples  from  this  distribution  co 
obtain  his  strategy  v. 

For  this  pure  strategy,  P.  say,  we  can  compute 

E,  ..Pr  (Kill  P  adopts  v  ;  E  adopts  v') 

(v  d  aiVy 

where  d  denotes  "is  distributed  as" 

These  quantities,  for  i  =  1.  .  .  n,  become  the  row  entries  in  the 
following  matrix  zero-sum  game: 


E 1  s 
strategies 
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The  solution  of  this  game  yields  the  optimal  strategy  for 
P,  and  the  optimal  strategy  for  E  which  will  consist  of  allocation 
of  effort  to  the  points  Ej, . E^  say. 

Note  that  E's  strategy  will  not  in  general  be  to  allocate 
effort  proportional  to  v'  . 

In  fact,  consider  the  numerical  example  used  previously. 
The  linear  programming  approach,  used  (for  simplicity)  with  a 
g(v)  of  the  form 

ao  +  al  v  +  a2  y2  +  a3v3 
and  with  strategies 

.1,  .2,  ....  1.0 

for  E,  yields  the  following  solution: 

P's  strategy:  aQ  =  1.3691  \ 

aj  =  -. 3192 

aE  =  -3.2451  |  9LP<V)#  Say 

a3  =  +3. 8591 


E's  strategy:  v' 


:  .025, 

Pr 

.0774 

.5 

Pr 

.0991 

.6 

Pr 

.4263 

1.0 

Pr 

.  3972 

Total 

1.0000 

This  strategy  for  P  results  in  the  kill  probabilities  shown  in 
column  (3)  of  Table  2. 

The  underlined  entries  in  this  column  indicate  how  E’s  strategies 
concentrate  on  P's  'weak  spots'.  In  fact,  since  P's  strategy,  because  of 
the  assumption  on  g(v),  results  in  approximately  Pr  (kill)  of  4th  degree 
polynomial  form,  it  is  sufficient  for  E  to  concentrate  his  efforts  on  four 
points:  one  at  the  boundary  v  =  1  where  P  (Kill)  is  decreasing  with  v 
(because  of  the  edge  effect  at  the  rim  of  the  speed  circle),  two  adjacent 
points  at  the  interior  minimum  which  occurs  between  v  =  .  5,  v  =  .6,  and 
one  at  the  center,  v  -  .025. 
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9 . 0  Efx eel  of  time  constraints  on  E's  strategies.  • 

(9*1)  Let  k(v)  =  Pr(Kill|E  adopts  speed  v,  P  adopts  a 

strategy  drawn  from  g*(v)  )  . 

Suppose  that  there  is  a  good  reason  for  E  ',  in  addition  to  avoiding 
detection,  to  get  as  far  away  as  possible  from  O  ,  in  a  stated  fixed 
time  interval. 

For  example,  it  might  be  known  (to  both  players)  that  additional 
P  forces  can  arrive  on  the  scene  in  1  hour's  time. 

To  pin  down  this  factor ,  let  us  define 

(9.2)  k„  (v)  =  Pr(Kill|E  adopts  speed  v  ,  no  kill  has  been  made 

£» 

by  time  t=l ,  P  adopts  some  optimal 
strategy  using  all  his  forces  after  time 
t=l) 

and  represent  k  (v)  by 

it 

(9.3)  k2(v)  =  X  (1-v2)  . 

Then  the  total  payoff  to  P  is 

(9  .4)  k^(v)  =  Pr(Kill  by  t=l )  t  Pr  (Kill  after  t=l ) 

=  k(v)  +  [1  -k(v)]  k2(v) 

=  k(v)  +  [l  -k(v)]  (l~v2)X  . 
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As  in  section  4,  we  require  for  an  optimal  solution,  that 

(9.5)  k^(v)  =  constant,  independent  of  v,  -  k  say  . 

Now,  an  approximate  expression  for  k(v)  is  available  in  eq. 
(4.13).  We  take  the  variable  K(v)  given  there  ,  and  multiply  by 
the  correction  factor  C  *  as  specified  in  eq .  (4.4)  et  seq .  ,  to 
obtain  k(v)  . 

This  expression  for  k(v)  may  be  substituted  into  (9.4),  and 

then  we  may  choose  the  constants  a  ,  a  ,  a  ,  a  ,  (a  chosen 

o  1  c,  3  3 

arbitrarily  to  be  -1),  so  as  to  satisfy  (9.5)  to  third  order  in  v  . 


1 0 .  Note  on  a  Smeared-Diagonal  Zero-Sum  Game 
10. 1  Purpose 

The  purpose  of  this  note  is  to  provide  analytical  support  for  an 
intuitively  obvious  fact. 

We  have  two  players  I  and  II ;  and  certain  physical  regions 

Rj  . .  .  in  which  they  can  allocate  'effort1  (e  .g  .  ,  for  a  search  game  , 

the  'effort'  will  be  the  percentage  of  time  they  are  in  region  R.).  The' 

amount  of  effort  will  be  described  by  q.  (Player  II),  and  p.  (Player  I), 

^  J 

and  the  pay-off  will  be 


q.  c..  P. 

i  ij  } 


to  player  II,  where  the  C..  are  (almost)  zero  unless  i=j  .  Our  heuristic 
reasoning  is  as  follows: 


Player  II  adjusts  his  q^  so  that  the  quantity 


V  q.  C.. 

Z_,  1 

i=l , . .  .n 


is  approximately  constant  (independent  of  j  ).  Then  it  is  claimed  (i)  that 
this  is  II' s  approximate  optimal  strategy,  and  (ii)  that  the  strategy  for  I 
which  consists  of  choosing  the  same  vector  for  the  p's  that  II  chose  for 
theq's.will  be  approximately  optimal. 
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Thic  claim  (i)  is  justified  for  the  model  o  *bc  paper  at,  folio,-. s: 
First,  by  the  Min-max  theorem,  exists,  sc  .itiens 

fa;}  »  (P;5  an^  a  constant  v  (the  value  ,  i  the  .  \ie  ch  that 

*  J 


i,  j  =  1, . . .  n 


q.  C.. 

^  L  i 


=  v 


Now 


V 


suppose  /  q.  C..  >  v  for  s-  e  j  ; 
Z— i  i  U 


Then  no  solution,  for  I,  ,  cji  .  ,  T  . 

However,  we  may  assume  from  the  ,;'i  ^  v 

that  it  would  not  be  reasonable  for  any  parti',  ■:  rpeeu  v  to  be 
unavailable  to  the  Evader  as  part  of  a  solution.  \«e  conclude  that 


g?me, 


any  solution  for  II  must  have^  q.  C.j  =  constant,  independent  of  j. 


10.  2  Application 

Apply  this  to  a  search  game  performed  over  the  unit  circle,  • 
divided  into  concentric  annuli  of  equal  width.  The  unit  circle  could  be  in 
speed  space  ,  or  in  real  space  but  in  any  case  we  require  that  the  sub- 
strategy  for  I  and  II  consisting  of  picking  a  particular  annulus  shall  result 
in  a  uniform  distribution  over  the  annulus  . 

A  pay-off  of  unity  ('success1')  occurs  ,  to  II,  if  the  two  players 
come  within  some  (small)  distance  of  each  other. 

Let  us  suppose  that  II  allocates  a  small  amount  of  effort  -  say  10%  -to 
a  particular  annulus  .  Assume  that  I  has  concentrated  all  his  effort  on 
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that  annulus  .  Then  the  probability  of  "success"  is  proportional  to 
l/(Area  of  annulus).  Thuo  II  allocates  his  effort  in  proportion  to  the  area 
of  the  annulus  ,  that  is, linearly  increasing  with  the  radius  of  the  annulus  . 
Similarly,  I  allocates  his  effort  in  the  same  proportions . 

10.  3  Detailed  Analysis 

Consider  the  following  zero-sum  game 


Player  II,  P 


q„  . 

ql-l 

qi 

qi+I  # 

41 

pl 

i 

pi-l 

1 

A. 

i-l 

C(i) 

A. 

l 

flayer 

I,  E  pi 

c(i-l) 

_1_ 

A. 

l 

€(i+l) 

A. 

l+l 

>-»* 

►— * 

f(i) 

A. 

i 

1 

A .  _ 
l+l 

• 
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The  constants  c{i)  are  supposed  small  ompared  to  1.  Also  the 
C (i )  are  supposed  to  vary  slowly  with  i  ,  and  so  are  the  A^  .  Consider 
mixed  strategies  for  I,  II  specified  by  {p^}  ,  {q.}  .  Then  the  payoff  (from 
II' s  point  of  view)  will  be 


n 


1 


Pi  < 


C(i  -  1) 


A. 


i-1 


Vl 


1 


+  ~-  q 
A.  ^i 

l 


+  iiLULL  > 

■  A  «i+l 


V 


say 


Suppose  we  can  find  a  strategy,  {q.  }  ,  say,  such  that  the  R.  aro  approxi- 

1 

mately  constant  (independent  of  i).  Then  we  assert  (i)  {q,  j  is  approximately 

♦  *  i 

the  optimal  strategy  for  II,  (ii)  the  strategy  fp^}  =  {q^}  is  approximately 
optimal  for  I.  (i)  is  obvious  ,  since  then  the  payoff  of  the  game  does  not 
depend  on  I's  strategy,  which  is  the  condition  we  have  assumed  for  optimality, 
(ii)  is  seen  as  follows: 

$  $ 

The  pay-off  with  the  suggested  strategy  {p^  }  a  (q^J  is 


Y  q  fp*  sfii  +  p*  jl  +  *  m  , 

h s  r:1Av  ;iA-  *»v 


S. ,  say 


But 


V 


<i 


*  iBl  +  *  Jl  *  *  id 

i-l  A.  si  A.  '  Hi+1  A. 

11  1 
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and  we  have,  since  R.  =  const,  for  q.  -  q.  , 

i  1  1 


J  *  SJLil+  *jl+  *  t G+lL 

'  I"'1  Ai-1  1  Ai  >+1  Ai+lJ 


const,  (indep.  of  i) 


Comparison  of  these  last  two  expressions  shows  that  S.  is  approximately 
independent  of  i  ,  and  thus  the  suggested  strategy  is  approximately  optimal. 
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x  x .  j.  a.- j.wo-atage  Spesd  Game 
11.0  Description  of  Model 

The  following  is  a  suggested  approach  to  solving  the  two- stage 
(for  E)  speed  game,  No  analytical  results  arc  obtained. 

The  same  initial  situation  is  assumed.  At  time  t=0,  an  Evader 
(E)  is  sighted  at  a  point  O  . 

Ke  realizes  he  has  been  sighted,  dives,  and  adopts  the  following 
two -stage  strategy: 

(i)  E  shooses  a  direction  at  random, 

(ii)  E  adopts  a  fixed  speed  w,  0  <  w  <  v^  , 

until  time  T  ,  then  adopts  a  fixed  speed  v,  0<v<vo  thereafter. 

A  Pursuer  (P)  is  located  at  distance  D  from  O  ;  we  suppose 
that  E  knows  P's  maximum  speed  and  distance  d  ,  but  not  his 
bearing . 

Previously  we  considered  strategies  for  P  which  consisted  of 
circles  in  the  'speed-space'  of  E  ,  corresponding  to  spirals  of  con¬ 
stant  pitch  in  real  space .  This  choice  is  no  longer  convenient. 

Instead  we  shall  consider  the  following  family  ^  say,  of 
strategies  for  P  .  P  steams  at  full  speed  (Q)  towards  O  ,  and 
stops  short  at  distance  r ^  (  >  0)  from  O  .  At  Bpeed  q  ,  he  then 
performs  search  around  a  circle  radius  r^  ,  center  O  .  Having 
completed  the  circle,  he  proceeds  along  the  radius  to  a  point  distance 
r 2  ,  and  performs  another  circle  in  the  same  sense  as  the  first,  and 
so  on. 

The  trips  between  successive  starting  points  of  the  circles  are 
made  at  speed  Q  ,  and  the  circles  themselves  with  speed  q  . 


, .  sa wwfwsssm’'*?'5*  '"*  ’  "'r "  ‘ 


The  same  “cookie -cutter"  radius  of  detection  of  E  by  P  will  be 
assumed;  however  the  radii  of  the  cookie -cutter  Rj ,  dependent 

on  the  respective  speeds  w,  v  of  E,  according  to  some  noise-law  to  be 
specified  . 

If  is  claimed  that  the  family  4^  is  "almost -dense"  or  "almost-good" 
in  the  sense  that  the  results  achieved  by  P  through  a  suitable  mix  of 
will  be  almost  as  good  as  those  that  could  be  obtained  with  any  general 
strategy.  The  objects  of  the  game  are,  for  E  ,  escape  without  detection, 
and  for  P  ,  detection  of  E  . 

It  is  conjectured  that  the  optimal  strategy  for  E  will  be  the  following 
(i)  Adopt  the  first  speed  w  up  to  time  T  ,  such  that  the  probability 
of  detection  by  T  (based  on  an  a  priori  knowledge  of  the  initial 
distance -fo-go  D  )  is  reasonably  small; 
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(ii)  Adopt  the  second  speed  v  by  sampling  from  a  p.d.f .  in  such 
a  way  that  the  distribution  ;  fixed  time  t  of  the  position  of  E  is 
approximately  uniform  over  some  circle  ,  or  some  area  formed 
by  subtracting  from  the  largest  circle  vQ.t  »  inner  circle  of 
some  radius . 

Since  the  speed  v  is  fixed,  once  chosen,  it  is  not  claimed  that 
this  distribution  can  necessarily  be  obtained  for  all  values  of  t  .  However, 
it  is  expected  that  the  true  distribution  will  be  approximately  uniform, 
for  each  of  a  wide  range  of  values  of  t  . 

11.1  Method  of  Analysis 

Regarding  the  initial  choice  of  radius  r ^  by  the  Pursuer.  We  may 
assume  that  r^  is  chosen  by  drawing  from  a  distribution  g(r)  -  on  analogy 
with  the  p.d .f.  g(v)  of  the  first  model. 

Regarding  the  second  and  subsequent  choices  of  radius  r  ,  we  may 
suppose  these  are  made  in  such  a  way  as  to  replicate  the  distribution  g(r)  . 
We  shall  ignore  the  effect  of  the  time  taken  to  move  between  successive 
circles  radii  r^  ,  r^  ...  . 

From  now  on,  we  may  concentrate  on  the  distribution  g(r)  . 

11.2  Choice  of  Evader  Strategies 

It  will  be  necessary  to  restrict  our  study  to  a  finite  number  of  pure 
strategies.  These  will  be  written 
(w,v,T) 

corresponding  to  the  strategy 

01.1) 


speed  w ,  0<  t  <  T 
speed  v,  t  >  I 


For  each  of  these  pure  strategics,  we  compute  the  probability  of  kill, 
thus 


First,  note  the  usual  correction  for  the  point  of  closest  approach  of  two 
bodies  in  (approximate)  straight -line  motion. 


Let  E’s  speed  be  generally  v  .  P’s  motion  relative  to  E  is  then: 


Kence  "kill"  occurs  if,  at  the  moment  P  crosses  the  path  of  E  ,  the 
distance  between  them  is  less  than 

(11.3)  x  =  R(1  +  v2/q2)1/Z  . 
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(Case  1)  Suppose  'kill1  occurs  during  the  'w'  of  first  phase  of  the 
(w,v,T)  policy.  We  shall  derive  the  range  of  the  argument,  (0j , 
such  that  Kill  results  if  E  should  choose  8  in  this  range,  as  follows: 
Suppose  that  'Kill'  occurs  on  the  first  circumnavigation  at  radius  r  , 
by  P  .  The  value  of  0^  is  given  by  the  following  consideration. 

P  must  be  at  point  A  when  E  is  at  point  A’  ,  precisely  dis¬ 
tance  x  closer  to  0  .  ♦ 


Thus,  provided  f>wT  4  x  , 


(11.4)  8j  is  given  by 


'¥]  - 


Similarly,  provided  r  >  wT  -  x  , 


(11.5)  is  given  by 


PS  *£l 


(Case  2) 

Suppose  "Kill'1  occurs  during  the  'Sr"  part  o£  the  (w,v,T)  strategy.  Then 
the  corresponding  equations  are: 


<“■«  h*  pif  +  ¥] 
«»■«  •i<-|^E+1rs] 


=  T  + 


=  T  + 


Note  that,  conceivably, solutions  in  excess  of  2  7J‘  may  arise  . 
(solutions  for  $  which  are  negative  will  be  rejected,  that  is  the  corresponding 
range  will  start  at  (:0). 

Note  also  that,  since  the  ranges  for  r  for  the  two  cases  - 
w-kill  and  v-kill,  overlap,  the  same  path  for  E  may  result  in  Kill  in 
both  cases . 


i"  v  '»  <.**_'  ‘  _ 

-  r  !■_  <  . 
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From  these  two  considerations  we  see  that  we  must  be  careful  to  ' 

avoid  counting  the  same  kill  twice. 

« 

Thus ,  the  ranges  for  0  must  be  added  together  on  an  ’or’  basis 
as  follows:  The  ranges  must  first  be  expressed  in  terms  of  coverage  of 
the  interval  [0,2 *■#*].  Then  Range  for  0  such  that  kill  results 

=  (Range  such  that  w-kiil  results) 

U  (Range  such  that  v-kill  results) 

On  dividing  the  result  by  2 Tf,  we  have  precisely  the  probability  defined  in 
(11.2),  integrand,  namely 

P(Killj  (w  ,v,T);  P's  strat.  *  r) 

On  integrating  with  respect  to  the  p.d.f.  g(r)  we  obtain 

P(Kill| (w,v,T);  P's  strat.  =  g(r)  ) 

It  should  then  be  possible  to  obtain  approximate  min-max  strategies  on  each 
side,  either  by  analytical  means,  or  by  a  Linear  Programming  approach,  as 
in  the  first  Model. 

The  resulting  value  of  the  game  (setting  T  =  »)  should  be  comparable 
with  the  value  for  the  first  game,  if,  indeed,  the  'circular'  strategies  are 
about  as  'good'  as  the  spiral  strategies  considered  in  the  first  Model. 
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12.0  To  some  extent,  we  may  approximate  the  two-stage  speed  game 
^  by  adapting  the  one-stage  speed  results  previously  given  (Model  I).  We 

consider  .the  same  spiral  strategies  for  P  as  were  used  in  Model  I). 

|  The  Evader  (E)  is  restricted  to  the  following  sub-set  of  two- 

stage  speed  strategies,  by  modifying  condition  (ii)  of  section  11.0 
^  .  (ii)  E  adopts  maximum  speed  v  for  time  Xt  ,  where 

I 

JJ  ‘  tQ  =  D/(Q  +vo)  ,  0<X<  1  , 

0 

||  then  adopts  fixed  speed  v,  0<  v<  v^,  thereafter. 

D 
D 
D 
D 
D 
I 
I 
I 
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Tht  constant  X  is  known  to  P  . 

Since  the  class  of  E-strategies  (ii )'  includes  the  class  used  in 
Model  1 ,  the  value  of  the  game  for  E  must  be  at  least  as  good  as  that  in 
Model  1 . 

12.1  Conclusions 

An  analytical  form  for  P's  optimal  strategy  is  exhibited . 

s 

12.  2  Technical  Discussion 


The  E-strategy  (v  ,  v)  is  seen  to  be  equivalent  to  a  single  speed 
strategy  v  ,  where  the  time -origin  is  backed-off  -Xt^  (v^/v  -1)  . 
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[ft  will  be  noticed  that  this  arrangement  is  not  defined  for  v  =  0  ;  since 
the  strategy  v  =  0  in  practice  falls  outside  the  scope  of  our  model,  it  will 
be  assumed  that  both  P  and  E  assign  zero  weight  to  this  strategy.] 

The  situation,  given  E  chooses  (v^,  v)  ,  is  thus  equivalent  to 
Model  1,  with  P  starting,  at  t  =  0  ,  at  a  distance 


D'sD+Q  .  X  t  <v  /v-1) 
o  o 


V  -V 
o 


D 


.DtQ.XC^-)  fgfc-) 


v  -  v  * 

=  D  [1  +  X  - -  .  -} 

Q  +  vq  v 


'Dt1^5fr)+Da5T7 


8 


where 


D*=  D  {1  -X  Q/(Q  +v  )} 

o 

H  “X  vq  Q/(Q  +  vo  -  X  Q) 


and  will  be  assumed  small 


Note  that ,  with  this  fictional  substitution  for  D  ,  the  analysis  of 
Model  1  carries  through  to  (and  including)  eq,  (4.14),  with  the  following 
exceptions : 

(1)  Equation  (3.2)  is  good  only  to  first  order  in  v  ; 

(2)  a  =  R/D  must  be  replaced  by 

a'  =  R/D*<1  v"1) 

.  >i<  - 1 

=  a  (1-/*  v  )  (first  order  in  )  . 

where  we  now  use  symbol  a  for  R/D  .  lb  first  order  in  /i  ,  we  may 
write 

•2  2  o 

a  =a  (1-2/iv  ),  etc . 

In  the  spirit  of  the  approximations  made  so  far ,  we  consider  the 
P-strategy  (c.f.  ,  oq.  (4.3)  ) 

*  22-1/2 
g  (v)  =  g(v)  .  v(q  -  v  ) 


where 


g(v)  =  ac  +  v  . 
Then,  from  eq.  (4.13), 


rf“  k(v)  =  (2  '  \*'A  +|  a’4)  (Q  +  v)  (a  +a  v) 
a  v  3  b  o  i 
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1  3  '3  2 

+  (-~a«  -  -a  )  (Q  +  v )  a  j 


where 


Hence 


=  [2  +  -  a  2  (1  -  2  |iv  1)  +  ~a.4(l-4fiy  1 )]  (Q  +  v)  (aQ  +  a  j  v) 

1  ±  1  O  jfcO  1  O 

+  C-  2  a  (I  -  M  v  )  -  ~ a  (1  -  3/iv"  )]  {Q  +  v)  a} 

-  [P  +  v"1  (-  ~a  2  -  |a*4)  /*]  (Q  +  v)  (ao  +  &1  v) 

+  [S  +  v"1  (i  a*  +  |  a*3)  n]  (Q  +  v)2  aj 

=  (P  +  G  v  *)  (Q  +  v)  (a  +  a  v) 

o  1 


+  (S+Hv4)  (Q  +  v)2  a, 


_  .  4  *2  8  *4, 

G  s  ("  T a  “  7*  )M  * 


1  *  Q  #3 

H=  (~a  +^a  *)#i  , 


„  2  *2  2  *4 

P=2+~a  + a  , 


_  1  *  3  *3 

S  =-?a  -ja 


4“  k(»)  ==  U  -fiv"1) 
a  v 


{v’1  [aQ  GQ  +aj  HQ2] 

+  [a  Q  P  +  a.  Q2  S  +  (a  +  a.  Q)  G  +  2  a,  Q  H] 
o  1  o  1  1 
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+  V  taQ  P  +  a1  Q  (P  +  2S)  +  a^  (G  +  H)] 

+  v2  [ax  (P  +  S)]  3  • 

(Assume  G,  H,  [i  are  small) 

4  v"1  [ao  G  Q  +  a2  H  Q2  -  /a  aQ  Q  P  -  /*  aj  Q2  S] 

+  [a  Q  P  +  a  Q2  S  +  (a  +  a.  Q)  G  +  2  a,  Q  H 
o  l  ol  1 

-/iaoP-fia1Q(P  +  2S)] 

+  v  [aQ  P  +  a}  Q  (P  +  2S)  +  &l  (G  +  H)  -  a}  n  (P  +  S)] 

+  v2  [*1  (P  +  S)]  . 

As  in  Model  I,  we  would  like  k(v)/a*v  to  be  independent  of  v  as  fg.r  fis  pos 
Since  the  coefficient  of  v  *  is  assumed  small,  this  suggests,  fol¬ 

lowing  Model  I,  we  make  the  coefficient  of  v  zero,  i.  e. ,  set 

a^  =  -  1  [arbitrary] 

Q  (P  +  2S)  +  G  +  T1  -  U  (P  +  S) 
aO  =  -  al  . . ~P .  • 
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The  corresponding  solution  to  the  same  accuracy,  in  Model  I  would  be 


Q(P  +  2*S) 

a  =  -  a,  - — - 

o  1  P 


[The  correction 

a  1  +  a  1=  1 
o  1  1  £ 

must  be  made  in  both  cases;  see  eq.  (4.4)]  .  However,  recall  that  P, 

S  are  defined  in  terms  of  a  not  a  and  a  differs  from  a  . 

Thus  the  optimal  strategy  for  P  is  modified. 
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13 


Consequences  of  non-optimal  policy  in  the  one-stage 
speed  game 


13.1  Introduction 

We  have  developed  optimal  policies  for  a  Destroyer  vs. 
Submarine  one-stage  speed  game.  The  question  has  been  raised: 

What  ard  the  consequences  should  the  Submarine  (Evader  ,  E)  adopt 
a  non-optimal  policy?  (In  particular,  if  he  adopts  maximum  speed  vq.) 
For  realism,  we  may  assume  E  chooses  a  speed  v  uniformly  in  the 
range  [v^  -  A,  v  ]  where  £(  =  0.1  vq,  say)  is  known  to  the  Destroyer 
(Pursuer,  P);  we  refer  to  this  as  the  near -maximum  speed  policy. 

13.2  Summary  of  Results 

For  the  numerical  example  used  ,  we  show  that  choice  of 
the  non-optirnal  near-maximum  speed  policy  by  E  leads  to  an  increase 
in  kill-probability  in  the  ratio  of  6:1  approximately. 

13.3  Analysis 


Suppose  E  adopts  a  speed  in  the  range 


(13.1)  [vq  -  A,  Vq] 


and  that  u  is  known  to  P.  Denote  the  annulus 

v  -A<  r  <  v  byA(A)  . 
o  —  —  o 

Then  P  adopts  a  "starting  point"  somewhere  in  the  same  range ,  which 
we  will  take  as  approximately  (vq  -  (1/2)  A  )  =  v 

The  time  of  arrival  of  P  at  v  is 


(13-2)  t2  = 

Q  +  v 

and  the  radius,  in  speed-space,  of  the  "cookie  cutter"  detector  is  R/ 1^ .  If 


(13.3)  R/t2  >  A  »  i*c*  t ^  <  R/A  =  t3»  say  then  only  width  A 

is  contributed  to  the  coverage  of  annulus  A  (A)-  This  continues  until 
t  =  tjj  then  the  "coverage"  is  R/t.  Thus  the  total  coverage  of  A  (A)  is 
approximately 


(13.4) 


J 


r  ,Z  2,  1/2  A 
1  (q  -  v  )  A 


dt  +  ^  (q  -  v 


2  2.  1/2  „  .. 

v  )  R  dt 


t  =  t. 


t  =  t. 


(13.5)  j(q2-vV/2  A  [log  (R/At2)  +  1]  if  t2  <  R/A 
1  * 

(13.6)  ^(q2  -  v2)  1/2  (R/t2)  ift2>'R/A 

13.4  Numerical  Example 

A  s  in  Section  6 ,  let  Q  =  2 . 0 

q  =  2.0 

R  =  0.05 

D  =  2.0  and  take  A  =  0 . 1 

Then  v  =  0.95, 

t2  =  D/(Q  +  v)  =  2/2.95  =  0.678  (>  R/u=  0.5) 

R/t  =  0.0737 

L* 

(q2-v2)l/Z  =  (3.0975)1/2  =  1.760 

Since  t2  >  R^we  use  formula  (13.6),  to  obtain:  total  coverage  of 
annulus  A  ( A  )  is 

(1.760)  (0.0737)  =  0.1298 

The  area  of  A(A)  is  tt  (l2  -  0 . 92)  =  tt. (0 . 1 9) 

Thus  P(Kill) .  2 tt  =  2(0 . 12981/0 . 19 

=  1.366 

In  Section  6,  when  E  used  optimal  strategy,  we  obtained 
P  (Kill) .  2 tt  =  0.221 

Thus  an  increase  in  the  kill-probability  in  the  ratio  of  6:1  is  observed. 


11-67 


tmmmsrr*9tmrn 


D 

D 

D 

D 

D 

fl 

I 

I 

I 

a 


Note  that  the  ratio  of  areas  of  the  speed  circle  utilized  by  E  in  these 
two  strategies  is 

2  2 

~ 1  -  0.19  =  1/5  approx. 

«  (1  > 

E  concentrates  his  effort  on  20%  of  the  available  "strategy-space"  (i.e. 
speed  circle). 

Why  is  P  able  to  improve  his  kill-probability  by  more  than 
the  5:1  ratio?  Because  P  is  relatively  a  more  efficient  searcher  near 
the  rim  of  the  speed  circle.  True,  his  transverse  speed  is  1 .760  compared 
to  2 .0  near  the  center  of  the  circle ,  but  his  radius  of  detection  in  speed 
space,  R/t,  is  0.0737  in  place  of  0.50  at  the  center  (where  t  =  1  hour 
approximately) . 

Unless  there  are  reasons  why  the  Evader  should  get  as  far 
away  as  possible ,  he  risks  a  6-fold  increase  in  P(Kill)  by  adopting  near¬ 
maximum  speed,  for  the  example  considered. 
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DISCRETE  TWO-PERSON  GAME  THEORY 
WITH  MEDIAN  PAYOFF  CRITERION 

ABSTRACT 

The  minimax  concept  for  solution  of  discrete  two-person  games  is 
based  on  expected  value  considerations  and  has  a  zero-sum  condition  for  . 
payoffs .  This  approach  often  is  inapplicable  due  to  strong  violation  of  the 
zero-sum  condition.  By  use  of  a  different  criterion,  based  on  median  value 
considerations,  a  two-person  game  theory  can  be  developed  that  seems 
appropriate  for  situation  which  we  call  competitive.  It  is  also  applicable 
for  a  much  broader  class  of  practically  important  situations  called  median 
competitive.  Consider  cases  where  each  player  is  either  protective  toward 
himself  or  vindictive  toward  the  other  player.  A  largest  value  P^  (Pjj) 
occurs  in  the  payoff  matrix  for  protective  player  I  (H)  such  that  he  can 
assure  himself  at  least  this  payoff  with  probability  at  least  50  percent.  A 
smallest  value  Pj.  (Pjj)  occurs  in  the  matrix  for  player  I  (Jt)  such  that 
vindictive  player  II  (I)  can  assure,  with  probability  at  least  50  percent,  that 
player  I  (H)  receives  at  most  this  payoff.  For  competitive  and  median  com¬ 
petitive  games,  a  player  is  simultaneously  protective  and  vindictive.  Values 
of  P  P  P'  P' 

I  ’  "  II  ’  I  ’  II  ,  and  median-optimum  strategieSjare  nearly  always 
determined  without  great  effort.  This  can  be  done  by  solution  of  zero-sum 
games  (expected  value  basis)  with  identified  payoff  matrices  containing  only 
ones  and  zeroes.  Deciding  on  payoff  values  is  simplified  for  the  median 
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approach.  Except  for  ,  Pjj  ,  P^1  ,  P^  ,  it  is  sufficient  to  know  the 
relative  order  of  the  values  for  each  payoff  matrix.  The  median  a;  jach 
has  the  strong  practical  advantage  of  being  applicable  even  when  payoffs 
in  different  matrices  cannot  meaningfully  be  added  or  subtracted  (such 
as  when  only  relative  ordering  is  known  for  one  or  both  matrices). 
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INTRODUCTION  AND  DISCUSSION 


This  paper  presents  a  form  of  discrete  two-person  game  theory 
that  is  based  on  median  value  considerations  (motivated  by  the  median 
estimation  concept  in  statistics).  Two  extreme  situations  arise,  depending 
on  whether  a  player  is  acting  protectively  for  himself  or  vindictively 
towards  the  other  player.  A  protective  player  is  interested  in  the  largest 
payoff  such  that  he  can  assure  himself  at  least  this  value  with  a  probability 
of  at  least  50  percent.  A  vindictive  player  is  interested  in  the  smallest 
payoff  such  that  he  can  assure,  with  a  probability  of  at  least  50  percent, 
that  the  other  player  receives  at  most  this  value.  A  class  of  "Median Optimum" 
strategies  can  be  defined,  for  either  the  protective  or  vindictive  approach. 

For  a  class  of  games  called  "competitive",  there  exist  strategies 
which  are  simultaneously  median  optimum  from  the  protective  viewpoint 
and  from  the  vindictive  viewpoint.  Consequently,  for  those  games, 
identification  of  a  median  optimum  strategy  as  protective  or  vindictive 
is  unnecessary.  In  spite  of  certain  parallels  between  the  median- optimum 
viewpoint  and  the  minimax  viewpoint,  a  pure  strategy  may  be  median 
optimum  without  necessarily  being  a  minimax  strategy. 

The  first  introductory  material  outlines  the  median  approach  and 
some  of  its  properties .  Then,  some  comparisons  are  made  with  the 
standard  minimax  form  of  two -per  son  game  theory. 
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Each  player  has  a  separate  matrix  that  states  the  payoff  he  receives 
for  each  combination  of  a  pure  strategy  of  his  with  a  pure  strategy  of 
the  other  player.  Both  of  these  matrices  are  considered  to  be  known  to 
each  player.  The  pair  of  payoffs  to  the  players  that  occurs  for  a  given 
combination  of  a  pure  strategy  for  each  player  is  an  outcome  of  the  game . 

The  first  value  in  an  outcome  is  the  payoff  to  player  I  and  the  second  value 
is  the  payoff  to  player  II. 

Each  player  then  imposes  a  preference-ordering  on  the  outcomes. 
The  preference -ordering  of  a  protective  player  I  is  considered  to  be  such 
that  his  own  payoffs  are  nondecreasing  and,  by  convention,  the  preference - 
ordering  of  the  protective  player  II  is  such  that  his  own  payoffs  are  non¬ 
increasing.  In  contrast,  a  vin  active  player  I  imposes  an  ordering  such 
that  the  payoffs  to  player  II  are  nonincreasing ,  and  a  vindictive  player  II 
imposes  an  ordering  where  the  payoffs  to  player  I  are  nondecreasing .  If 
there  are  tied  values  among  the  payoffs  considered,  there  may  be  many 
alternative  orderings  satisfying  any  one  of  those  conditions.  A  vindictive 
player  could  order  within  ties  so  as  to  be  as  advantageous  to  himself 
as  possible .  A  protective  player  could  order  within  ties  so  as  to  be  as 
disadvantageous  to  the  other  player  as  possible.  On  the  other  hand,  a 
protective  player  could  order  within  tk  •  so  as  to  be  as  advantageous  to  the 
other  player  as  possible,  etc.  Finally,  more  than  one  outcome  could 
possibly  have  the  same  pair  of  payoffs .  A  player  may  select  among  the 
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possible  orderings  of  such  "double  ties"  on  the  basis  of  the  strategy  com¬ 
binations  corresponding  to  these  outcomes.  In  all  cases,  each  player 
(identified  as  protective  or  vindictive)  chooses  a  sequence  that  is  called 
his  preferred  sequence.  The  preferred  sequences  provide  the  basis  for 
application  of  the  median  approach. 

A  game  is  said  to  be  competitive  if  the  outcomes  can  be  sequence 
ordered  so  that  the  payoffs  for  player  I  are  nondecreasing  and  the  payoffs 
for  player  II  are  nonincreasing.  The  possible  ordering  of  outcomes  is 
unique  when  the  payoffs  of  player  I  are  strictly  increasing  or  those  of 
player  II  are  strictly  decreasing;  then  we  assume  that  the  preference 
orderings  of  the  two  players  are  the  same  .  However,  more  than  one  eligible 
'sequence  order  is  possible  when  ties  in  payoff  value  occur  for  both  placers . 
That  is,  the  same  outcome  value  could  possibly  occur  for  more  than  one 
combination  of  strategies.  Among  these  sequences  (which  are  the  same 
in  values  of  outcomes),  each  player  selects  his  preferred  sequence  on  the 
basis  of-the  strategy  combinations  corresponding  to  the  pertinent  tied 
outcomes  and  related  probability  considerations.  If  the  preferred  sequence 
is  still  not  unique  ,  it  can  be  chosen  arbitrarily. 

A  largest  payoff  (P^)  occurs  in  the  matrix  for  player  I  (II)  such 
that  he  can  assure  at  least  this  value  with  a  probability  of  at  least  50  per¬ 
cent.  A  smallest  payoff  Pj  (Pjj)  occurs  in  the  matrix  for  player  I  (II) 
such  that  player  II  (I)  can  assure ,  with  a  probability  of  at  least  50  percent, 
that  player  I  (II)  receives  at  most  this  amount. 


Let  ua  consider  determination  of  (Pjj)  for  a  protective  player 

I  (U).  There  exists  a  subset  o'.  outcomes  in  the  preferred  sequence  for 
protective  player  I  (II)  that  consists  of  a  determined  outcome  and  all 
outcomes  above  (below)  it.  One  of  the  outcomes  in  this  identified  subset 
can  be  assured  with  a  probability  of  at  least  50  percent,  but  such  is  not 
the  case  when  the  determined  outcome  is  removed.  A  protective  player  is 
considered  select  his  preferred  sequence  so  as  to  minimize  the  number  of 
outcomes  in  the  identified  subset.  When  more  than  one  eligible  subset 
has  the  minimum  number  of  outcomes,  a  subset  that  has  the  highest 
assured  probability  is  used  for  the  preferred  sequence.  The  value  of 

Pj  (Pjj)  i®  the  payoff  for  player  I  {  II)  in  the  lowest  (highest)  outcome  of 
his  identified  subset.  Incidentally,  one  or  more  outcomes  consecutively 
adjacent  to  the  identified  subset  could  also  have  payoff  (P^)  for 
player  I  (II). 

Now,  consider  determination  of  the  payoff  P'  (Pj.)  ,  in  the 
other  player’s  matrix,  that  is  associated  with  a  vindictive  player  I  (II). 
There  exists  a  subset  of  outcomes  in  the  preferred  sequence  for  vindic¬ 
tive  player  I  (II)  that  is  identified  in  the  same  way  as  was  given  for 
evaluation  P^  and  P  .  The  value  of  P^  (PJ)  is  the  payoff  to  player 

II  (I)  in  the  lowest  (highest)  outcome  of  the  identified  subset  for  player  II  (I). 

Detailed  statement  of  results  for  the  general  case  occurs  in  the 
next  section.  However,  some  additional  information  about  results  for 
competitive  games  is  given  here.  When  each  player  has  a  pure  median 
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optimum  strategy  i  player  I  (1J)  can  guarantee  (100  percent  probability)  at 

least  P.  =  PI  (PTT  =  P')  .  When  player  I  (II)  has  a  pure  median  optimum 

1  1  II  11  • 

strategy  but  player  II  (I)  does  nC:,  player  I  (II)  turn  guarantee  at  least 
P  (Pjj)  *nd  player  II  (I)  can  assure  at  icatt  Pp  (PJ)  with  a  probability 
greater  than  50  percent.  When  neither  player  has  a  pare  median  optimum 
strategy,  player  I  (II)  can  assure  that  receives  at  least  P^  (P^)  with 
probability  at  least  50  percent,  and  that  player  II  (I)  receives  at  most 
P‘j  (Pp  with  probability  at  least  50  percent . 

It  is  instructive  to  consider  some  characteristics  of  the  median 
of  a  probability  distribution  for  a  situation  where  both  pla*  se  mixed 
strategies.  The  median  value  of  a  distribution  is  not  net  tarlly  unique. 
That  is ,  all  the  permissible  values  in  an  extensive  interval  car  be  medians 
of  a  probability  distribution.  This  property  can  be  convec.^ut.  Suppose 
that  the  game  is  competitive  and  consider  the  set  of  median  values  for 
player  I  (II).  Also,  suppose  that  both  players  use  mixed  strategies  that 
are  optimum  for  the  median  approach.  Then,  all  payoffs  that  are  at 
least  equal  to  Pj  (Pjp  and  at  most  equal  to  P^  (P^)  medians  of  the 
probability  distribution  of  the  payoff  to  player  I  (II).  Thus,  as  would  be 
expected  in  competitive  situations,  player  I  (II)  seeks  to  maximize  the 
upper  payoff  P^  (P^)  in  his  set  of  median  values  and  to  minimize  the  lower 
payoff  Pp  (Pp  in  the  set  of  median  values  for  player  II  (I).  The  properties 
of  a  median  allow  this  to  be  done  simultaneously  in  such  a  way  that 
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Pj  (Pjj)  and  PJ  (PJj)  can  be  *ar  apart  in  an  ordering  of  the  payoff  values 
for  player  I  (II).  It  is  thus  possible  for  Pj  to  be  in  the  uoper  payoff 
values  for  player  I  simultaneously  with  Pjj  being  in  the  upper  values  for 
player  II.  In  fact,  this  seems  to  occur  for  many  kinds  of  combinations  of 
payoff  matrices  for  players  I  and  II  in  competitive  games. 

Required  information  about  payoff  matrices  is  not  very  fjreat  when 
the  median  approach  is  used.  It  is  sufficient  to  first  determine  the  rela¬ 
tive  order  (includes  equality)  of  the  values  for  each  matrix,  which 
determines  the  locations  of  P_,  Pi,  P„,  and  Pi.  .  Deciding  on  values 
for  Pj,  PJ,  Pjj,  and  Pjj  completes  the  information  that  is  required 
libout  the  payoff  matrices  . 

Perhaps  the  most  attractive  feature  of  the  median  approach  is  its 
ability  to  handle  competitive  games  where  payoffs  from  different  matrices 
cannot  be  meaningfully  added  or  subtracted .  This  permits  use  of  a  game 
theory  approach  for  an  important  and  extensive  class  of  situations  .  In 
fact,  many  of  the  situations  occurring  in  economics,  and  the  social 
science  areas  (psychology,  education,  etc.)  have  matrices  of  this  kind, 
maybe  due  to  the  fact  that  only  relative  order  can  be  determined  within  a 
payoff  matrix.  However,  situations  of  this  class  occur  in  virtually  all 
areas  where  game  theory  is  potentially  useful  (including  military  appli¬ 
cations  } 


It  is  more  difficult  to  determine  the  appropriat  ness  of  the  median 
approach  when  the  game  is  not  even  roughly  of  .  competitive  nature. 

Then,  a  low  payoff  for  one  player  does  not  necessarily  correspond  to  a 
high  payoff  for  the  other .  Thus ,  the  median  payoff,  say  to  player  I, 
might  be  substantially  different  for  the  protective  and  vindictive  situa¬ 
tions  .  Also ,  cases  where  cooperation  would  increase  the  payoff  to  both 
players  can  occur  .  However  ,  the  median  approach  can  be  useful  when 
the  players  are  not  allowed  to  communicate  (so  that  a  player  only  knows 
his  own  payoff  matrix).  Also,  results  like  those  developed  for  competitive 
games  can  be  obtained  for  a  rather  broad  class  of  situations  that  are 
termed  median  competitive . 

The  most  general  form  of  median  competitive  games ,  and  cor¬ 
responding  properties,  have  not  been  determined  yet  and  provide  a  subject 
for  future  investigation.  An  example  of  games  that  are  median  competitive , 
but  not  necessarily  competitive ,  is  given  here.  The  example  consists  of 
all  games  that  ’'generate"  competitive  games .  A  game  is  said  to  generate 
a  competitive  game  if,  for  both  players  ,  there  exist  sequences  that  are 
eligible  to  be  preferred  sequences  and  for  which  the  following  two  condi¬ 
tions  are  satisfied:  First,  the  payoffs  of  player  I  (II)  that  are  in  outcomes 
above  (below)  the  outcome  determining  (P  )  are  at  least  (most)  equal 
to  Pj  (Pjj)»  and  the  payoffs  in  outcomes  below  this  outcome  aie  at  most 
(least)  equal  to  P^  (P^).  Second,  the  payoffs  of  player  I  (II)  that  are 
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below  (above)  the  outcome  determining  PJ  (P^)  are  at  least  (most)  equal 
to  P^  (Pjj)  »  and  the  payoffs  in  outcomes  below  this  outcome  are  at  most 
(least)  equal  to  PJ  (P^)  •  Then,  new  outcomes  can  be  formed,  by  pairing 
the  payoffs  of  player  I  with  those  of  player  II ,  that  satisfy  the  requirements 
for  a  competitive  game  but  leave  the  outcomes  that  determined  P^,  P^ ,  PJ, 

Pjj  fixed  and  at  the  same  sequence  positions.  This  is  done  so  that  the 
groups  of  payoffs  in  the  identified  subsets  for  the  competitive  game  are  the 
same  as  the  groups  in  these  subsets  for  the  original  game  .  Since  the 
results  developed  depend  only  on  the  outcomes  that  determine  P^,  P^,  P^, 

PJj  and  on  the  groups  of  payoffs  in  the  identified  subsets ,  the  results  for 
this  competitive  game  also  apply  to  the  game  from  which  it  was  generated. 

Now  let  us  compare  the  median  approach  with  the  minimax  procedure 
where  a  zero-sum  condition  is  imposed  on  the  two  payoff  matrices  and  the 
criterion  is  the  expected  value  of  the  payoff  to  player  I.  Discussion  of 
expected  value  and  median  value  properties  occurs  first. 

The  outcome  that  results  when  one  or  both  players  use  a  mixed 
strategy  is  a  random  value  .  This  outcome  can  be  identified  by  a  repre¬ 
sentative  property  of  its  probability  distributions  .  The  mean  of  this 
distribution  (expected  value  of  the  random  outcome)  is  one  representative 
property  that  could  be  considered.  The  distribution  median  (not  necessarily 
unique)  is  another  representiative  property  that  is  useful.  Each  of  these 
properties  has  desirable  and  undesirable  features.  Neither  has  been  shown  to 
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be  uniformly  preferab’e  to  the  other .  Usually,  choice  of  whether  to  consider 
the  expected  value  or  the  median  value  is  based  on  the  utility  aind  conven¬ 
ience  aspects  of  the  situation.  If,  for  the  situation  considered,  much  more 
extensive  results  can  be  obtained  for  the  median,  statisticians  seldom 
hesitate  to  consider  it  rather  than  the  distribution  mean. 

Discrete  two-person  game  theory  is  a  case  where  median  consider¬ 
ations  seem  to  lead  to  more  extensive  results  of  a  worthwhile  nature  than 
do  expected  value  considerations .  The  median  approach  handles  the 
rather  broad  class  of  competitive  and  median  competitive  games,  including 
games  where  one  or  both  payoff  matrices  have  ordinal  members.  The 
minimax  approach  requires  cardinal  members  in  both  matrices  but  still 
only  applies  to  the  small  subclass  of  competitive  games  where  the  matrices 
at  least  roughly  satisfy  a  zero-sum  condition.  Values  must  be  determined 
for  all  (or  nearly  all)  of  the  outcomes  when  the  minimax  approach  is  used. 
Except  for  a  few  payoffs  (usually  four,  and  never  more  than  four)  only 
relative’ order  among  the  payoffs  in  each  matrix  must  be  determined  for 
the  median  approach. 

For  games  of  a  zero-sum  type  ,  it  would  seem  that  a  combined  use 
of  the  expected  value  criterion  and  the  median  approach  could  be  desirable  . 
That  is,  the  strategy  used  by  a  player  is  at  least  approximately  optimum 
in  '.n  expected  value  sense  and  also  assures  at  lea<?^,  an  identified  payoff 
with  a  probability  that  has  a  lower  bound  not  greatly  below  50  percent. 
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The  resulting  median  payment  would  ordinarily  be  less  than  for  player 
I  and  less  than  Pjj  for  player  II.  Such  strategies  are  especially  desirable 
when  values  of  payoffs  are  only  roughly  known  but  relative  ordering  is 
precisely  known  within  each  matrix.  The  determination  of  strategies  with 
these  combined  properties  is  another  subject  for  future  investigation. 

Only  discrete  games  are  considered  here.  However,  extension 
of  the  median  approach  to  continuous  cases  ,  and  combinations  of  continu¬ 
ous  and  discrete  cases,  seems  definitely  possible  and  worthwhile  .  This 
extension  is  a  further  subject  for  future  investigation. 

The  next  section  contains  statements  of  how  to  determine  P^»  Pj., 

P  ,  P'  ,  and  optimum  strategies  for  each  player.  Also,  properties  of 
results  using  the  median  approach  are  stated  more  precisely.  The  final 
section  contains  the  basis  for  the  results  (in  terms  of  three  theorems). 
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Let  the  payoff  matrix  for  player  I  (II)  be  stated  so  that  rows 
represent  pure  strategies  for  player  I  (II)  and  columns  are  pure  strategies 
for  player  II  (I).  For  all  applications  ,  a  marking  of  some  of  the  values 
in  the  payoff  matrices  is  made  initially,  with  this  being  done  separately 
for  each  matrix.  The  case  of  a  protective  player  is  considered  first. 

For  protective  player  I  (II),  first  mark  the  position,  in  his  matrix, 
of  his  payoff  i  i  the  last  (first)  outcome  of  his  preferred  sequence  of  out¬ 
comes  .  Then  do  this  for  the  next  to  the  last  (first)  outcome  for  player 
I  (II),  etc.  Continue  consecutively  in  his  preferred  sequence  of  outcomes 
until  the  first  time  that  this  player  can  assure  obtaining  a  marked  value 
with  probability  at  least  1/2.  The  value  of  (P^)  i8  the  last  payoff 
marked  ii.  the  matrix  for  player  I  (II).  For  competitive  games  ,  (Pjj)  is  the 
payoff  to  player  I  (II)  in  the  last  outcome  that  was  marked  in  the  matrix 
of  player  II  (I). 

Determination  of  P^  (P^)  >  and  the  corresponding  pairs,  is  perhaps 
best  accomplished  by  initially  marking  the  matrix  for  player  I  (II)  until 
the  first  time  that  two  or  fewer  rows  contain  marks  in  all  the  columns . 

(The  value  of  P^  (P^)  is  greater  than  or  equal  to  the  last  payoff  marked 
in  this  manner,  and  can  be  greater;  based  on  Theorem  1  .)  Next,  working 
forward  (backward)  in  the  preferred  sequence  for  protective  player  I  (II), 
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remove  the  mark  from  the  payoff  (unique)  that  was  marked  last.  Then, 
replace  the  remaining  marked  values  with  ones  and  replace  all  other 
payoffs  in  the  matrix  by  zeroes.  Consider  this  matrix  of  ones  and  zeroes 
to  be  for  a  zero-sum  game  with  an  expected  value  basis .  Solve  for  the 
value  of  the  game.  If  this  game -value  is  less  than  1/2,  the  marking  is 
completed  by  again  marking  the  payoff  whose  mark  was  removed  (which 
also  determines  the  corresponding  pair).  If  the  game-value  is  at  least 
1/2 ,  continue  in  the  same  way  (removing  the  mark  from  the  last  payoff 
that  was  considered  among  those  still  marked,  forming  a  matrix  w-ith 
ones  and  zeroes,  etc.).  If  the  resulting  game-value  is  less  than  1/2 
the  payoff  whose  mark  was  last  removed  is  marked  again  and  the  marking 
is  completed.  This  marking  procedure  is  continued  until  a  game -value 
less  than  1/2  occurs  .  (This  procedure  ,  and  that  in  the  next  paragraph, 
are  based  on  Theorem  2.)  From  the  examples,  it  seems  that  and 
Pjj  are  often  the  payoffs  that  resulted  in  the  first  time  that  two  or  fewer 
rows  coiitain  m?rked  values  in  all  columns  of  the  respective  matrices. 

The  zero-sum  game  (matrix  of  ones  and  zeroes)  that  occurs  for 
the  final  marking  in  evaluating  P^  or  P^  is  also  used  to  determine 
(protective)  median  optimum  strategies  for  the  player  with  that  matrix. 
That  is ,  an  optimum  strategy  of  this  player  for  that  game  is  also  a 
median  optimum  strategy.  In  particular ,  consider  the  situation 
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for  player  I  (II)  when  (Pjj)  happens  to  be  the  payoff  whose  marking 
resulted  in  a  pair  of  rows  that>contain  marked  values  in  all  columns 
(but.  no  fully  marked  row  occurs).  Examination  of  the  zero-sum  game 
shows  that  a  mixed  median  optimum  strategy  for  player  I  (II)  consists 
in  choosing  one  of  the  rows  of  this  pair  with  probability  1/2  for  each  row. 

For  player  I  (II)  vindictive  ,  first  mark  the  position  in  the  matrix 
for  player  II  (I)  that  is  in  the  last  (first)  outcome  of  the  preferred 
sequence  of  player  1  (II).  Then  do  this  for  the  next  to  last  (first)  outcome 
for  player  I  (II).  Continue  consecutively  in  the  preferred  sequence  for 
player  I  (II)  until  the  first  time  that  he  can  assure  obtaining  a  marked 
value  in  the  matrix  of  player  II  (I)  with  probability  at  least  1/2  .  The 
value  of  (Pj)  is  the  last  payoff  marked  in  the  matrix  for  player  II  (I). 

Determination  of  Pj^  (Pj)  can  be  accomplished  by  initially 

marking  the  matrix  for  player  II  (I),  according  to  the  vindictive  preferred 

sequence  for  player  I  (II),  until  the  first  time  that  two  columns  contain 

marks  in  all  the  rows  .  Next,  remove  the  mark  from  the  payoff  that  was 

marked  last.  Replace  the  remaining  marked  values  with  ones  and  all 

other  payoffs  by  zeroes.  Consider  the  resulting  matrix  to  be  for  a  zero- 

sum  game  and  solve  for  the  game -value  .  If  this  game -value  is  greater 

than  1/2,  the  marking  is  completed  by  again  marking  the  payoff  whose 

mark  was  removed.  If  the  game -value  is  at  most  1/2,  continue  in  the 

same  way  with  removal  of  another  mark.  If  the  resulting  game -value 

is  greater  than  1/2,  again  mark  the  payoff  whose  mark  was  last  removed 
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and  the  marking  is  completed.  This  marking  procedure  is  continued  until 
a  game-value  greater  than  1/2  occurs.  As  for  the  protective  case,  it 
seems  that  P^  and  PJj  are  often  the  payoffs  that  resulted  the  first 
time  that  two  or  fewer  columns  contained  marked  values  in  all  rows . 

The  zero-sum  game  that  occurs  for  the  final  marking  of  the  matrix 
fox*  player  II  (I)  can  be  used  to  determine  (vindictive)  median  optimum 
strategies  for  player  I  (II).  That  is ,  an  optimum  strategy  of  player  I  (II) 
for  this  game ,  that  is  based  on  the  matrix  for  player  II  (I),  is  also  a 
median  optimum  strategy .  When  P^  (Pp  happens  to  be  the  payoff  that 
resulted  in  a  pair  of  columns  with  marks  in  all  rows  (but  no  fully  marked 
column  occurs),  a  mixed  median  optimum  strategy  for  player  I  (II) 
consists  in  selecting  one  of  these  two  columns  with  probability  1/2  for 
each  column. 

Statement  of  results  occurs  next.  Cases  where  pure  median 
optimum  strategies  occur  are  considered  first.  In  all  cases,  a  protective 
player  I  (II)  can  guarantee  himself  at  least  P^  (Pjj)  by  using  the  fully 
marked  row  in  his  matrix.  A  vindictive  player  I  (II)  can  always  guarantee 
that  player  II  (I)  recieves  at  most  Pj^  (Pj,),  by  using  the  fully  marked 
column  in  the  matrix  for  player  II  (I). 

Now,  consider  the  case  where  each  player  has  .a  pure  median 
optimum  strategy.  When  a  protective  player  I  (II)  uses  the  fully  marked 
row  in  his  matrix  and  a  vindictive  player  II  (I)  uses  the  fully  marked  column 


in  the  matrix  for  player  I  (II),  player  I  (II)  receives  exactly  P  = 

(Pjj  =  PJ  )  and  player  II  (I)  receives  the  payoff  in  his  matrix  that 

corresponds  to  the  strategy  combination  for  this  row  and  column. 

When  protective  players  I  and  II  both  use  fully  marked  rows,  player  I 

sometimes  receives  more  than  P^  and/or  player  II  sometimes  receives 

more  than  P  .  When  vindictive  players  I  and  II  both  use  fully  marked 
IX  ** 

columns  (in  the  other  pHyer's  matrix),  player  I  sometimes  receives  .. 

\ 

less  than  P^  and/or  player  II  sometimes  receives  less  than  Pjj  . 

When  the  game  is  competitive  and  each  player  has  a  pure  median  optimum 
strategy,  P^  =  P^  and  P^  =  P’^  also,  when  each  player 

uses  his  pure  median  optimum  strategy,  player  I  (II)  receives  P^  (Pjj). 

Now,  consider  the  case  where  a  pure  median  optimum  strategy 
occurs  for  player  I  (II)  but  not  for  player  II  (I).  First,  suppose  that 
player  I  (II)  is  protective .  Then,  player  I  (II)  can  guarantee  himself  at 
least  Pj  (Pjj)  .  and  a  protective  player  II  (I)  can  assure  himself  at 
least  Pjj  (Pj)  with  a  probability  of  at  least  1/2  .  Player  I  (II)  can  also 
gxiarantee  himself  at  least  P^  (P^)  against  a  vindictive  player  II  (I),  and 
player  II  (I)  can  assure  that  player  I  (II)  receives  at  most  P^  =  P^  (P^  =  Pjj) 
with  a  probability  greater  than  1/2  (Theorem  3).  Next,  suppose  that 
player  I  (II)  is  vindictive.  Then,  player  I  (II)  can  guarantee  that  a  protective 
player  II  (I)  receives  at  most  Pj,j  =  Pjj  (Pj.  =  P^),  and  player  II  (I)  can  assure 
himself  at  least  P^  (P^)  with  a  probability  greater  than  1/2  (Theorem  3). 
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Player  I  (II)  can  also  guarantee  that  vindictive  player  II  (I)  receives  at 
most  PJ^  (PJ)  ,  and  player  II  (J)  can  assure,  with  probability  at  least 
1/2,  that  player  I  (IX)  receives  at  most  PJ  (Pjp  .  Now  consider  competitive 
games.  Then,  Pj  =  PJ  and  Pjj  =  PJ^  (Theorem  3).  Player  I  (II)  can 
guarantee  that  he  r'eceives  at  least  P^  (P  )  and  that  player  II  (I)  receives 
at  most  Pjj  (Pj)  .  Player  II  (I)  can  assure  that  he  receives  at  least 
Pjj  (Pj)  :  also  that  player  I  (II)  receives  at  most  P^  (P  )  with  a  probability 
greater  than  1  /2 . 

Finally,  consider  the  case  where  no  pure  median  optimum  strategy 
occurs  for  either  player.  Suppose  that  both  players  are  protective.  Then, 
player  I  (II)  can  assure  at  least  P^  (Pjj)  with  a  probability  of  at  least 
1/2 .  When  player  I  (II)  is  protective  and  player  II  (I)  is  vindictive  , 
player  I  (II)  can  assure  that  he  receives  at  least  Pj  (Pn)  with  probability 
at  least  1/2  and  player  II  (I)  can  assure  that  player  I  (II)  receives  at  most 
Pi  (py  with  probability  at  least  1/2;  these  probabilities  are  exactly  1/2 
when  both  players  use  mixed  median  optimum  strategies.  Next,  suppose 
that  both  players  are  vindictive .  Then,  player  I  (II)  can  assure  that  player 
H  (I)  receives  at  most  PJj  (PJ)  with  a  probability  of  at  least  1/2.  Now 
consider  competitive  games.  Player  I  (II)  can  simultaneously  assure , 
with  probability  at  least  1/2,  that  he  receives  at  least  Pj  (P^)  and  that 
player  II  (I)  receives  at  most  PJ^  (PJ)  .  When  both  players  use  mixed 
median  optimum  strategies,  player  I  (II)  receives  at  least  Pj  (P^)  with 
probability  exactly  1/2  and  at  most  PJ  (PJj)  with  probability  exactly  1/2. 
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does 


Sometimes  ,  a  bound  (for  payoff  values,  that  can  be  guaranteed 
not  differ  much  from  the  bound  (determined  using  the  median  approach, 
that  is  only  assured  with  a  probability  of  at  least  ,/*•  the  guaranteed 

bound  would  often  be  preferred.  Existence  of  such  situations  should  be 
considered  in  applications  of  the  median  approach  (for  payoff  matrices 


with  cardinal  numbers). 
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BASIS  FOR  RESULTS 


The  procedure  for  determining  P|,  Pj.^,  and  vindictive  median 
optimum  strategies  can,  with  suitable  interpretation  be  obtained  directly 
from  that  for  determining  Pj,  P^,  and  protective  median  optimum 
strategies.  Hence,  verification  for  the  protective  case  is  sufficient. 

The  results  for  both  players  protective,  or  both  vindictive,  can 
be  directly  verified  from  the  properties  of  the  procedures  for  determining 
Pj,  Pjj,  PJ,  and  Pjj  .  This  is  also  the  case  for  one  player  protective 
and  the  other  vindictive  when  both  players  have  pure  median  optimum 
strategies  or  neither  player  has  a  pure  median  optimum  strategy. 

A  competitive  game  can  be  considered  to  be  a  combination  of  the 
situation  where  player  I  is  protective  and  player  II  vindictive  with  the 
situation  where  player  I  is  vindictive  and  player  II  protective .  Thus  ,  to 
verify  properties  of  competitive  games,  it  is  sufficient  to  present  proof 
for  the  pertinent  case(s)  of  one  player  protective  and  the  other  vindictive. 
Finally,  it  is  to  be  noted  that  competitive  players  can  have  different 
preferred  sequences  in  the  sense  of  different  combinations  of  strategies 
being  associated  with  outcomes  that  have  the  same  value.  However,  this 
causes  no  difficulties  in  derivations  since  the  preferred  sequences  are 
the  same  with  respect  to  the  values  of  the  outcomes. 
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The  following  three  theorems  contain  the  verification  that  is  not 

evident  from  the  properties  of  the  procedures  for  determining  P  ,  P  , 

•  1  XX 

PJ,  and  PJj. 

Theorem  1  .  The  procedure  of  marking  payoffs  (in  his  matrix) 
for  a  player  until  the  first  time  that  two  or  fewer  rows  contain  marked 
values  in  all  columns  guarantees  that  occurrence  of  a  marked  value  can 
be  assured  with  a  probability  of  at  least  1/2. 

Proof:  First  note  that  continued  marking  ultimately  results  in 

this  situation.  When  one  row  becomes  fully  marked,  the  probability  is 
unity  that  some  one  of  the  marked  values  can  be  assured  by  the  player. 

Next,  suppose  that  a  pair  of  rows  is  needed.  When  two  mixed 
strategies  p^ ,  . .  . ,  p^  and  ,  . .  .  ,  are  used  (pure  strategies  are 
special  cases,  and  there  are  r  rows  and  s  columns),  the  probability 
of  the  marked  subset  is 

l  P.O.. 

i.i  ’  1 

where  is  the  sum  of  the  q's  for  columns  that  have  marked  payoffs 

in  the  i-th  row.  The  largest  value  of  this  probability  that  the  player  can 

assure  (by  choice  of  p  ,  .  . .  ,  p  )  is 

1  r 


mm 

qr  ••••  qs 


(max.  Q.) 

l  l 
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Let  i(l)  and  i(2)  denote  two  rows  that  together  contain  marked  payoffs 
in  all  columns .  For  any  minimizing  set  of  q's  .both  Q.^  and  Q.^j 
are  at  most  G  ,  so  that 

and  a  probability  of  at  least  1/2  can  be  assured.  This  probability  can 

exceed  1/2  but  is  exactly  1/2  when  the  unmarked  payoffs  are  such  that 

two  columns  contain  unmarked  payoffs  in  all  rows  (since  analogously, 

the  set  of  unmarked  payoffs  can  be  assured  with  a  probability  of  at  least 

1/2).  It  is  also  exactly  1/2  when  there  are  two  columns  that  have  an 

unmarked  payoff  in  one  of  the  two  rows  and  are  such  that  no  row  of  the 

matrix  has  payoffs  marked  in  both  columns . 

* 

Theorem  2  .  A  lower  bound  on  the  probability  that  a  player  can 
assure  one  of  a  specified  subset  of  outcomes ,  and  corresponding  optimum 
strategies ,  can  be  determined  by  solution  of  a  zero-sum  game  with  an 
expected  value  basis  .  The  payoff  matrix  for  this  zero-sum  game  has 
ones  at  the  positions  that  correspond  to  the  (pure)  strategy  combinations 
for  the  subset  of  outcomes,  and  zeroes  elsewhere. 

Proof;  Let  each  player  use  an  arbitrary  mixed  strategy  (a  pure 
strategy  occurs  as  a  special  case).  The  expression  for  the  expected  pay¬ 
off  of  the  zero-sum  game  is  also  the  expression  for  the  probability  that 
some  one  of  the  outcomes  in  the  specified  subset  occurs. 


Theorem  3 .  When  protective  player  I  (II)  has  a  fully  marked  row 


in  his  matrix,  but  vindictive  player  II  (i)  does  not  have  a  fully  marked 
column  in  this  matrix,  PJ  =  (P^.  =  P  )  ;  also,  player  II  (I)  can  assure 

that  player  I  (II)  receives  at  most  PJ  (JP^)  with  a  probability  greater 
than  1/2  .  Likewise  ,  when  vindictive  player  I  (II)  has  a  fully  marked 
column  in  the  matrix  for  protective  player  II  (I),  but  player  II  (I)  does 
not  have  a  fully  marked  row  in  this  matrix,  PJj  =  P^  (Pj  =  P^)  ;  also, 
player  I  (II)  can  assure  himself  at  least  Pjj  with  a  probability  greater 
than  1  /2 . 

Proof:  Consider  the  outcome  that  corresponds  to  the  last  payoff 
marked  for  protective  player  I  (II)  and  the  outcomes  that  do  not  correspond 
to  marked  payoffs  for  player  I  (II).  This  set  of  outcomes  can  be  assured 
with  probability  greater  than  1/2  by  vindictive  player  II  (I).  Otherwise, 
player  I  (II)  would  have  terminated  his  marking  procedure  earlier .  The 
payoffs  for  player  I  (II)  in  this  set  of  outcomes  are  at  most  equal  to 
PJ  (PJj)  »  with  equality  holding  for  the  outcome  corresponding  to  the 
last  payoff  marked  for  player  I  (II).  This  follows  from  the  development 
of  preferred  sequences  for  competitive  games.  Also,  since  player  I  (II) 
has  a  fully  marked  row  in  his  matrix,  player  II  (I)  cannot  assure  that 

9 

player  I  (II)  receives  any  payoff  less  than  PJ  with  nonzero  probability. 

Thus,  P.  =  Pi  (PTT  =  Pi.)  and  player  II  (I)  can  assure,  with  a  probability 
a  I  II  II 

greater  than  1/2,  that  player  I  (II)  receives  at  most  P^  (Pj^)  • 


*■  ~~  -n  *** 
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A  similar  verification  can  be  given  for  the  case  of  vindictive 
player  I  (II)  having  a  fully  marked  column  and  protective  player  II  (I) 
not  having  a  fully  marked  row. 


APPENDIX:* 

EXAMPLES  OP"  THE  MEDIAN  PAYOFF  CRITERION  AS  APPLIED  TO 
ZERO-SUM  MATRIX  GAMES 


1.  The  preceding  paper,  "Discrete  Two-Person  Game  Theory 

with  Median  Payoff  Criterion,"  by  John  E.  Walsh,  introduced  the 
concept'  of  a  median  payoff  criterion  for  a  two-player  matrix  game  . 
The  present  paper  is  intended  to  clarify  th»t  concept  by  providing 
some  examples ,  which  will  be  modifications  of  some  well  known 
matrix  games,  including  "Matching  Pennies"  and  "Paper,  Stone,  and 
Scissors 

2  .  Consider  the  game  ,  "Matching  Pennies  ,"  whose  matrix  is 

as  shown.  (We  will  use  A  =  (a..)  as  the  original  payoff  matrix.) 


As  usual,  the  payoff  matrix  shows  payments  from  Player  II  to  Player  I. 
Thus,  Player  I  is  the  maximizing  player,  and  controls  the  row-index  i 
of  the  matrix;  Player  II  is  the  minimizing  player  and  he  controls  the 
column -index  j  of  the  matrix. 

In  general,  neither  Player  I  nor  Player  II  will  have  a  strategy 
which  is  under  all  circumstances  better  than  ("dominates")  his  other 
strategy.  Thus ,  in  general,  each  of  the  players  must  play  a  mixed 
strategy,  which  will  prevent  his  opponent  from  predicting  his  behavior 
with  certainty.  In  fact,  the  ordinary  game -theoretic  minimax  solution 
to  the  game  whose  matrix  is  shown  above ,  is  that  each  player  should 


*  --  by  John  P.  Mayberry 


play  each  of  his  strategies  with  probability  1/2.  The  result  is  an 
expected  payoff  of  1/2;  i  .e .  ,  denoting  the  minimax  value  of  the  game 
by  v(A),  we  see  that 

v(A)  =  1/2 

Note  that  the  expected  value  of  1/2  is  obtained  by  averaging  outcomes 
which  must  be  either  =  0  or  =  1;  consequently,  the- probability  of  an 
outcome  equal  to  1  must  be  1/2,  if  both  players  play  optimally  in  the 
minimax  sense . 

With  these  strategies,  in  other  words,  the  minimizing  player 
has  guaranteed  that  he  will  achieve  an  outcome  as  low  as  0  with 
probability  1/2  --  while  the  maximizing  player  has  simultaneously 
assured  that  he  will  obtain,  with  probability  1/2,  an  outcome  as  great 
as  1.  This  example  illustrates  a  case  where  the  median  -  optimum 
value  of  the  game  is  not  uniquely  defined,  but  is  an  entire  interval  -- 
here  the  interval  between  0  and  1 . 

\ 

3 .  Suppose  now  that  we  modify  the  payoffs  in  the  game  of  "Matching 

Pennies"  somewhat,  so  that  Player  I,  who  is  trying  to  match,  wins  only 
3/4  when  the  match  is  on  "Heads)'  but  continues  to  win  1  when  the  match 
is  on  "Tails  ."  The  matrix  A  then  appears  as  follows: 


which  is  smaller  than  1/2, 
of  the  outcomes  to  the 

maximizing  player . 


The  minimax  value  of  this  game  is  now  3/7, 
as  it  should  be  since  we  have  decreased  one 
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However,  the  median -optimum  solution  strategies  remain 
unchanged  --  each  player  should  still  play  his  two  alternatives  with 
probability  1/2  each.  The  median -optimum  value  of  this  game  is 
now  the  interval  from  0  to  3/4,  instead  of  the  interval  from  0  to  1  .  If 
the  payoff  in  one  of  the  non-match  cells  (a  or  a,  )  had  been  increased  , 
in  addition  to  the  decrease  in  one  of  the  matching  cells,  the  interval 
which  represented  the  median -optimum  value  would  have  been  further 
narrowed,  but  the  median-optimum  strategies  would  have  remained 
unchanged. 

If  we  consider  a  new  example ,  in  which  the  decreasing 
"match"  payoff  and  the  increasing  "non-match"  payoff  ar?  almost 
equal,  we  might  get  a  matrix  such  as  this: 


So  long  as  (  >  0  there  will  be  an  interval  of  uncertainty  for  the  median 
value,  and  the  median -optimum  strategies  will  be  unchanged.  If  e 
becomes  <  0,  however  ,  it  will  happen  that  one  player  (in  this  case  ,  I) 
will  have  a  strategy  (in  this  case  ,  his  second)  which  dominates  his 
other  strategy,  so  that  the  game  will  be  strictly  determined.  The 
strictly-determined  game  will,  of  course,  have  an  optimal  pure 
strategy  in  the  usual  minimax  sense  and  the  same  will  necessarily 
be  true  in  the  median -optimum  sense  . 

(This  last  fact  can  easily  be  seen  by  noting  that  the  presence 
of  a  saddle  point,  which  is  equivalent  with  the  conditions  that  the  game 
be  strictly  determined,  identifies  a  value  such  that  each  player  can 
guarantee  that  the  payoff  is  at  least  as  favorable  to  him  as  that  value 
with  probability  equal  to  1  and  not  merely  with  p  obability  equal  to  1/2; 
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I 

I 

I 

I 

I 

I 
D 
0 
D 
D 
D 
D 
D 
D 
U 
0 

II 


consequently,  each  player  will  possess  a  median -optimum  strategy 
which  consists  of  the  pure  strategy  identified  by  that  saddle  point, 
and  the  median -optimum  value  will  be  equal  to  the  minimax  value  . ) 

In  fact,  there  are  only  two  possibilities  for  a  2-by-2  median- 
optimum  game:  either  the  game  is  strictly  determined,  in  which  case 
its  median -optimum  value  is  equal  to  its  minimax  value;  or  the  game 
is  not  strictly  determined,  in  which  case  its  median  value  will  be 
the  whole  of  an  interval  between  the  Becond- smallest  and  the  second- 
largest  of  the  four  entries  in  the  matrix. 

4 .  Let  us  consider  now  the  game  whose  matrix  is  shown  here: 

Vo  o  i / 

This  is  the  game  of  the  "pea  under  the  walnut."  Player  I  is  trying  to 
guess  under  which  walnut  the  pea  was  left,  and  Player  II  is  attempting 
to  put  the  pea  under  one  of  the  three  walnuts  so  that  Player  I  does 
not  guess  correctly.  Ignoring  the  possibility  of  sleight-of-hand,  which 
makes  this  game  so  interesting  to  professionals,  we  find  the  minimax 
value  to  be  1/3 ,  and  the  optimal  strategies  for  each  player  are  the 
same  --to  play  each  strategy  with  probability  1/3.  But  since  the  only 
outcomes  possible  are  0  and  1 ,  the  fact  that  Player  I*s  minimax- 
optimal  strategy  gives  him  an  expected  value  of  1/3  means  that  he 
cannot  play  so  as  to  achieve  a  payoff  =  1  with  probability  greater 
than  1/3;  consequently  0  is  the  largest  number  which  Player  I  can 
assure  that  he  will  attain  with  probability  1/2.  Consequently,  the 
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median-optimum  value  of  this  game  is  =  0.  The  presence 
of  the  l's  in  the  matrix  is  useless  to  the  maximizing  player,  since 
he  has  no  strategy  which  would  permit  him  to  achieve  those  out¬ 
comes  with  probability  1/2. 

5.  It  is  important  to  notice  that  the  first  and  last  of  the  above 

examples  have  a  significant  distinguishing  property,  apart  from 
their  symmetry  --  namely,  that  the  possible  outcomes  are  only  0 
and  1 .  Consider  now  a  general  matrix  A,  with  entries  either  0  or 
1;  this  matrix  defines  a  game  whose  outcomes  consist  of  only  0  and 
l's.  The  minimax  value  v(A)  of  that  game  must  be  either  less  than 
1/2,  equal  to  1/2,  or  greater  than  1/2. 

If  v(A)  =1/2,  then  Player  I  must  possess  a  mixed  strategy 
which  guarantees  to  him  an  expected  value  of  at  least  1/2,  regard¬ 
less  of  what  Player  II  does  .  But  the  only  way  to  achieve  an  expected 
value  of  at  least  1/2,  when  the  only  outcomes  possible  are  0  and  1 ,  is 
to  achieve  the  outcome  1  with  probability  at  least  1/2.  The  same 
argument  shows  that  the  minimizing  player  can  achieve  an  outcome 
of  0  with  probability  at  least  1/2.  Asa  consequence,  the  whole 
interval  between  0  and  1  represents  the  set  of  median -optimum  values 
of  the  game . 

If,  instead,  v(A)<  1/2,  then  Player  II  must  have  a  strategy 
that  guarantees  an  expected  outcome  of  less  than  1/2;  this  implies 
that  an  outcome  =  0  be  achieved  by  II  with  probability  greater  than 
1/2,  regardless  of  Player  l's  choice  of  strategy.  Consequently, 
the  median- optimum  of  the  game  must  be  given  by  vmQ  =  0. 
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Finally,  if  v(A)  >  1/2,  the  median-optimum  of  that  game 

must  be  given  by  v  =  1 . 

mo 

It  is  obvious  that  those  three  cases  exhaust  all  the  possibilities 

« 

for  matrix  games  with  entries  either  0  or  1 . 

6.  With  regard  to  the  question  of  numerically  calculating  the 

median -optimum  payoff  for  a  given  matrix  game  whose  outcomes  are 
not  restricted  to  the  values  Oand  1,  two  approaches  may  be  taken.  The 
first  is  conceptually  much  clearer  and  reveals  the  essence  of  the 
situation;  the  second  promises  to  be  more  useful  for  computational 
purposes.  In  this  section  we  describe  the  conceptually  preferable 
approach . 

If  a  number  s  is  a  median -optimal  value  of  a  matrix  game 
defined  by  a  matrix  A  ,  then  Player  II  can  assure  that  (with  probability 
at  least  1/2)  no  outcome  >  s  will  be  obtained,  while  Player  I 
can  assure  that  (with  probability  at  least  1/2)  no  outcome  <  s 
will  be  obtained.  Let  us  define  a  derived  matrix  game  A'  =  (a. ' . ^ , 
whose  entries  are  given  by 


(Let  us  suppose ,  for  the  moment,  that  there  are  no  entries  a„  =  s; 
the  contrary  case  will  be  considered  below.)  Now,  Player  II  can, 
in  the  derived  game ,  so  play  as  to  insure  that  the  probability  of  a  0 
will  be  at  least  1/2,  while  Player  I  can  so  play  that  the  probability 
of  a  1  is  at  least  1/2.  Consequently,  we  must  have  v(A')  =1/2.  But 
if  that  condition  holds  for  the  value  s ,  which  was  by  assumption  not 
equal  to  any  entry  in  the  payoff  matrix,  it  must  hold  for  an  entire 
interval,  ranging  at  least  from  the  matrix  entry  next  lower  than  s 
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to  the  matrix  entry  next  higher  than  s . 

We  summarize  the  characteristics  of  the  case  considered 
above  as  follows:  The  entries  in  the  original  payoff  matrix  may  be 
divided  into  two  setsi which  we  can  call  the"low  set"and  theMhigh  set,JI 
each  element  of  the  low  set  must  be  smaller  than  each  element  of  the 
high  set;  when  the  elements  of  the  low  set  are  replaced  by  0  and  the 
elements  of  the  high  set  are  replaced  by  1  ,  the  minimax  value  of  the 
resulting  matrix  game  is  1/2;  in  this  case  ,  any  number  between  the 
greatest  payoff  in  the  low  set  and  the  smallest  in  the  high  set  will  be 
a  median -optimum  value  for  the  original  game;  it  is  not  excluded 
that  the  value  may  actually  range  over  a  wider  interval,  if  some  of 
the  largest  elements  of  the  low  set  of  outcomes,  and/or  some  of  the 
smallest  elements  of  the  high  set  of  outcomes,  actually  have  no 
effect  on  the  minimax  value  of  the  derived  game . 

In  short,  we  see  that  it  is  possible  to  determine  the  median 
payoff  value  of  a  matrix  game  by  considering  a  graph,  like  Figure  1 
on  th  e  facing  page ,  which  shows  for  each  proposed  median-optimum 
value  s  (plotted  on  the  abscissa)  the  minimax  value  v(A')  of  the  derived 
game  which  would  result  if  we  replace  all  payoff  entries  smaller  than 
s  by  0 ,  and  all  payoff  entries  larger  than  s  by  1 .  Of  course ,  when  s 
is  very  small,  all  the  entries  of  the  derived  matrix  A1  will  be  =  1  , 
and  the  minimax  value  of  the  derived  game  will  also  be  =  1;  wher  j  is 
very  large ,  all  the  entries  in  the  derived  matrix  A*  will  be  =  0,  and 
the  minimax  value  of  that  derived  game  would  be  =  0.  No  change  can 
occur  in  the  mlnimax  value  of  the  derived  game  except  when  the 
proposed  s  traverses  one  of  the  entries  in  the  original  payoff  matrix, 
at  which  time  v(A')  may  change  .  The  graph  must,  therefore  ,  consist 
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least  entry 
in  A 


8  -* 


greatest  entry 
in  A 


FIGURE  I 

Showing  Minimax  Value  v(A')  of  the  Derived  Game  A'  for 

Various  s 


of  a  non-increasing  function  in  a  series  of  descending  steps,  each 
representing  an  interval  in  which  v(A‘)  does  not  change . 

If  one  of  those  steps  happens  to  be  at  the  height  1/2,  we 
have  the  situation  described  above,  where  partitioning  the  elements 
of  the  game  matrix  into  a  high  set  and  a  low  s^t  will  provide  a  game 
whose  minimax  value  is  1/2.  It  can  very  well  occur,  howevei  ,  that 
none  of  those  horizontal  steps  is  at  height  1/2.  Then  consider  the 
last  step  greater  than  1/2,  and  the  next  following  step  (which  is 
less  than  1/2).  (Since  the  first  step  is  preceded  by  an  interval  within 
which  the  minimax  value  is  =  1 ,  and  the  last  step  is  followed  by  an 
interval  within  which  the  minimax  value  is  =  0,  there  must  be  such  a 
step.) 

There  will  necessarily  then  be  a  unique  s,  separating  the 
minimax  values  greater  than  1/2  from  those  less  than  1/2.  That  value 
s  must  be  the  median -optimum  value  of  the  original  matrix  and  will 

be  denoted  by  v  (A ) . 

1  mo' 

7.  Recall  the  concepts  of  minorant  and  majorant  games,  as 

introduced  by  von  Neumann  and  Morge.istern  in  Ref  [l].  The  majorant 
game  is  the  game  in  which  Player  II  first  chooses  one  of  his  pure 
strategies,  and  his  decision  is  revealed  to  the  maximizing  player,  who 
then  riiakes  his  decision.  This  information  gives  a  potential  advantage 
to  I,  and  the  value  of  the  majorant  game,  when  only  pure  strategies  are 
permitted  to  each  player,  represents  an  upper  bound  on  the  reasonable 
"values"  for  the  game  .  This  value  is  denoted  by  v  (A),  and  is  given  by 

v  (A)  =  min  max  a., 

J  i 

Because  Player  II  can  assure  that  the  expected  payoff  will  not  exceed 
v,  Player  II  can  never  be  forced  to  tolerate  a  higher  value  than  v. 
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Analogously,  we  find  the  minorant  game ,  whose  value  v 

is  given  by  v  =  max  min  a..;  v  represents  a  lower  bound  on  what 
i  j  lJ 

Player  I  could  ever  be  forced  to  tolerate . 

If  v  =  v,  the  game  is  strictly  determined,  and  possesses  a 
saddle  point;  each  player  can  choose  an  optimal  pure  strategy  and  the 
value  of  the  game  v(A)  is  the  common  value  of  v  and  v. 

v(A)  =  v(A)  =  v(A). 

This  argument  is  entirely  ordinal  --  that  is  to  say,  it  depends  only 
on  ordinal  relations  among  the  elements  in  the  payoff  matrix,  and 
not  on  any  geometric  or  arithmetic  relations  among  them.  Consequently, 
the  median  payoff  criterion  could  be  applied  to  a  matrix  game  which 
possesses  a  saddle  point,  and  could  not  influence  the  decision  of 
rationality;  each  player  should  still  play  for  the  saddle  point. 

Therefore,  if  the  game  possesses  a  saddle  point,  the  median- 
optimal  value  will  be  equal  to  the  usual  minimax  value  ,  which  will  be 
equal  to  the  value  at  that  saddle  point.  The  converse  is  not  true;  there 
exists  a  game  which  has  no  saddle  point,  but  whose  median- optimum 
value  is  equal  to  one  of  the  entries  in  the  matrix.  An  example  is  given 
by  this  matrix: 

A  = 

which  is  ,  in  fact,  simply  the  game  of  Stone  ,  Paper  ,  and  Scissors  ,  or 
Jan-Ken-Pon.  The  median -optimum  value  o *  this  game  is  0,  because 

-1  <  s  <  0  implies  v(A')  =  1/3,  while 
0  <  s  <  1  implies  v(A')  =  2/3  . 

If,  on  the  other  hand,  v  <  v,  then  an  examination  of  the  argument 
of  section  6  above,  shows  that  the  median -optimum  value  must  lie  between 
them: 

v  <  v  <  v . 

—  -  mo  - 
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8.  Finally,  we  present  three  other  examples;  the  first  and 

third  were  generated  by  entering  random  numbers  in  a  6-by-6  matrix. 
The  first  has  numbers  to  three  decimal  places,  and  therefore,  does 
not  have  any  "ties"  between  entries  in  its  payoff  matrix.  Under  these 
circumstances  ,  it  is  still  an  open  question  whether  the  median  “Optimum 
values  will  occupy  an  entire  interval  whenever  the  matrix  does  not  have 
a  saddle  point.  The  second  example  was  obtained  by  selecting  numbers 
from  0  to  5  at  random,  and  therefore  contains  many  duplications  among 


its  outcome  values  .  The  median -optimum  payoffs ,  the  minimax  payoffs  , 
and  the  optimal  strategies  are  shown  in  each  case . 


The  first  example  is  given  by  this  matrix: 


.130 

.611 

.895 

.731 

.398 

.856, 

.065 

.084 

.894 

.738 

.971 

.  636 

.781 

.004 

.222 

.431 

.064 

.980 

.855 

.053 

.472 

.265 

.985 

.044 

.904 

.051 

.216 

.597 

.202 

.261 

‘.685 

.060 

.863 

.667 

.336 

.364' 

\ 


/  • 


For  this  game ,  the  minorant  value  v  is  given  by 

v  =  max  min  a. .  =  0.130, 
i  j 

and  the  minorant  value  v  is  given  by 
v  =  min  max  a..  =  0.611 . 

j  i 

The  median -optimum  strategies  £mo  and  T)mo  are  given  by 


f 


mo 


,  0,  0,  1/2,  0,  0), 


T)  n  =  (*  1/2,  0,  0,  0,  0), 

'  mo 


and  the  median -optimum  value  consists  of  the  interval  from  0.13C  )  0.611: 


v  =  (0.130,  0.611)  . 
mo 
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In  the  above  example ,  v  is  the  entire  interval  between  v 

mo 

and  v.  That  is  not  the  case  in  general,  and  we  present  a  modifica¬ 
tion  of  the  above  example  to  illustrate  the  point  (a^  has  been  decreased 


from  0.985  to 

0.385): 

. 

.130 

.611 

.895 

.731 

.398 

.856 

^.065 

.084 

.894 

.739 

.971 

.636  ^ 

.781 

.004 

.222 

.431 

.064 

.980  \ 

A  = 

.855 

.053 

.472 

.265 

.385 

.044  1 

1-904 

.051 

.216 

.597 

.202 

.261  / 

'.685 

.060 

.863 

.667 

.336 

.364' 

For  the  above  matrix  A,  it  is  still  true  that 

v  =  min  max  a..  =  0.611 ,  and  that 
j  i 

v  =  max  min  a  .  =  0. 130; 

“  i  j  il 

and  the  median -optimum  strategies  are  also  unchanged: 

£  =  (1/2,  0,  0,  1/2,  0,  0), 

S  mo 

•» 

•  n  =  (1/2,  1/2,  0,  0,  0,  0). 

mo 

However,  the  median -optimum  value  Is  now  a  narrower  range: 

v  -  (0.130,  0.398). 
mo 

It  is  worth  noting  that  a  median -optimum  matrix  game  (with 
determinate  entries  in  the  payoff  matrix),  even  when  zero-sum,  has 
some  of  the  characteristics  of  a  non-zero-sum  game.  This  happens 
because  the  median -optimum  value  may  (and  in  general  will )  be  different 
for  the  two  players;  in  this  last  example,  Flayer  I  can  assure  that  the 


payoff  is  at  least  0.398  with  probability  at  least  1/2,  while  Player 

« 

II  can  assure  that  the  payoff  is  at  most  0.130  with  probability  at 
least  1/2.  It  will,  in  some  cases,  be  possible  for  "collusion"  to 
take  place  between  the  players;  the  ramifications  here  seem  to  be 
very  complex . 

Another  consequence  of  t  Jemaaks  is  that  a  median- 
optimum  game  cannot  be  reducer1  rnei^pts  a  minimax- optin 

game  can,  Ref  [2] .  If  we  examine  ^bmatjp:  of  the  above  exar 


rnef^pt8  a  minimax- optimum 
.bmatjp:  of  the  above  example 


which  consists  of  those  entries  in  this  pay  off  Batrix  which  are  chosen 
with  non-rero  probability  in  some  pair  of  median -optimum  strategies, 
we  get  the  2-by~2  submatrix 


L11  ai2\ 
41  a42  / 


.611  \ 
.053/ 


'  41  42  /  V -  '  —  i 

in  both  cases;  the  median -optimum  values  of  this  2-by-2  matrix  game 

are  (0.130,  0 . 61 1 )  in  both  cases  .  In  the  second  example  ,  in  other 

words ,  the  median -optimum  values  are  not  the  same  for  that  sub¬ 


matrix  as  for  the  whole  matrix. 


If  there  exists  a  concept  in  the  theory  of  median -optimum 
games,  which  corresponds  to  the  "kernel"  of  a  minimax  game,  it 
seems  that  it  will  have  to  be  a  more  complex  notion  than  the  original 
notion  of  kernel  presented  in  Ref  [2] . 

For  our  thir  a  example ,  we  consider  the  matrix  A  given  here; 
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In  this  matrix. 


v  =  max  min  a..  =  2  , 
i  j  1J 

v  =  min  max  a..  =  3  , 

j  i  1J 

t  n  =  (0,  1/2,  0,  0,  1/2), 

5  mo 

n  =  (0,  0,  1/2,  0,  1/2,  0),  and 
mo 

V  *  <2’  3)‘ 
mo 

Note  that  this  game  can  be  reduced  to  a  kernel;  the  matrix 


a  a  > 
23  25 


a63  a65 


3  1 


2  5 


has  median-optimum  values  between  2  and  3 ,  just  as  the  whole  original 
matrix  did . 

The  minimax  strategies  are  given  by 

23f  =  (0,  11,  0,  0,  2,  10), 

23r)  =  (0,  8,  12,  0,  3,  0), 

and  the  value  by 

63 

.  v(A )  =  — 


9.  A  significant  question  on  the  subject  of  median-optimum 

games  remains  open.  If  the  entry  in  row  i,  column  j,  of  the  outcome 

matrix  is  a  random  variable  x„  having  a  continuous  probability- 

distribution  (with  E(x. .)  »  a..), rather  than  a  determinate  outcome, 

ij  ij 

then  the  derived  matrix  A'  will  be  given  by 

a'..  =  prob  fthe  random  variable  x. .  exceeds  s ]  . 
ij  * 


Since  each  a'.,  will  then  be  a  continuous  ,  strictly-decreasing  function 
of  s ,  and  since  v(A')  is  a  continuous  function  of  the  a'.. ,  it  follows  that 


v(A')  is  a  continuous  strictly-decreasing  function  of  s  . 


Thus ,  there  will  be  a  unique  value  of  s  for  which  v(A')  =1/2  and 


v  (A )  is  unique  . 
mo 

Computation  of  in  this  case  is  another  open  question. 
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MIXED  STRATEGIES  IN  PRACTICE 


I.  INTRODUCTION 

The  notion  of  mixed  strategy  has  been  of  fundamental  importance 
in  the  study  of  zerc-sum  two-player  games  ever  since  it  was  first  intro¬ 
duced  by  Borel  (ref.  [1]),  and  has  been  the  subject  of  much  discussion, 
both  in  a  formal  mathematical  context  and  in  an  informal  practical  con¬ 
text.  Much  of  the  mathematical  argument  has  focused  on  the  convenience 
and  logical  necessity  of  the  concept,  while  much  of  the  informal  discus¬ 
sion  has  emphasized  the  impracticality  of  a  mixed  strategy  in  applications. 
(A  common  objection  may  be  expressed  along  these  lines:  *'  practical 
man  will  never  believe  that  he  ought  to  make  rational  decisions  by  flipping 
a  coin".)  The  two  principal  theses  of  this  paper  are,  first,  that  the  mathe¬ 
matical  purposes  for  which  the  mixed  strategy  was  designed  can  be  fulfilled 
by  a  variety  of  practical  means,  apart  from  actual  randomization:  and, 
second,  that  a  minimax  solution  (even  if  it  is  not  a  pure  strategy)  can  be, 
and  often  is,  put  into  effect  by  a  practical  player. 

Although  we  assume  that  the  reader  has  some  familiarity  with  the 
basic  notions  of  two-person  zero-sum  games,  we  shall  briefly  recapitulate 
the  historical  development  of  the  mixed  strategy  to  define  our  viewpoint, 
and  shall  then  proceed  to  suggest  how  the  purposes  of  the  mixed  strategy 
can  be  achieved  without  "coin-flipping". 

These  suggestions  lead  naturally  to  further  suggestions  on  the  use 
of  specific  techniques  10  achieve  the  benefits  of  minimax  solutions,  without 
the  inconvenience  of  randomization.  These  conclusions  and  recommenda¬ 
tions  appear  to  be  of  considerable  potential  importance  in  a  wide  class  of 
practical  problems. 
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II.  HISTORICAL  SUMMAPY 


The  principal  purpose  which  is  achieved  by  the  device  of  the 
mixed  strategy  is  to  provide  to  a  participant  an  explicit  recommendation 
for  action,  which  nevertheless  does  not  permit  that  action  to  be 
predictable. 

In  the  case  of  "Matching  Pennies"  ,  the  recommendation  states  that 
each  competitor,  ("p**,yer")  ought  to  choose  heads  and  tails  equally  often, 
and  without  pattern  -  in  other  words,  ought  to  choose  each  alternative  at 
random,  with  equal  probabilities.  It  is  certainly  sufficient  to  achieve  the 
minimax  expectation  if  the  selection  of  pure  strategies  is  made  at  random. 
However,  it  is  not  necessary  that  the  stiategies  be  chosen  at  random,  but 
merely  that  they  be  chosen  in  a  way  which  cannot  be  predicted  by  the 
opponent.  In  fact,  whenever  one  player  attempts  to  implement  a  mixed 
strategy,  he  necessarily  takes  a  "random  number"  (or  more  precisely  a 
"pseudo-random  number"  as  generated  by  some  device)  uniformly  distrib¬ 
uted  between  0  and  1,  and  then  chooses  (for  example)  heads  whenever  that 
random  number  is  less  than  one-half,  tails  otherwise.  If  that  pseudo¬ 
random  number  is  known  to  the  opponent,  then  this  mixed -strategy  recom¬ 
mendation  will  also  be  transparent  to  the  opponent,  and  the  first  player's 
actions  will  in  fact  be  predictable  by  the  opponent. 

It  appears  that  the  critical  consideration  is  not  whether  the  choice 

44 

is  act  lally  random  ,  but  whether  it  can  be  observed  or  inferred  by  the 


In  the  version  of  "Matching  Pennies"  discussed  by  Von  Neumann 
and  Morgenstern  (ref.  [2]),  each  player  chooses  either  "Heads"  or  "Tails"; 
player  I  wins  a  penny  from  player  II  if  the  choices  match,  pays  a  penny  to 
player  II  if  they  do  not. 

One  might  pursue  a  complex  side  issue  at  this  point,  discussing 
the  question  of  whether  using  a  specified  random  sequence  requires  an 
additional  randomization  to  determine  which  elements  of  the  sequence 
(e.  g  ,  numbers  below  one-half  a"  against  numbers  above  one-half)  should 
correspond  to  tails.  One  might  also  discuss  the  question  of  whether  or  not 
the  opponent  would  know,  even  if  he  had  access  to  the  random  sequence, 
just  how  it  was  being  used  -  so  that  he  could  employ  that  knowledge  to  pre¬ 
dict  your  behavior.  These  points,  although  of  some  interest,  are  not  ger¬ 
mane  to  our  principal  concern  in  this  paper. 
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opponent.  A  deterministic  sequence  is  quite  satisfactory  if  we  can  be  cer¬ 
tain  tnat  the  opponent  does  not  know  it,  and  a  random  sequence  is  totally 
unsatisfactory  if  the  opponent  can  discover  it. 
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III.  THREE  ASPECTS  OF  A  SPECIFIC  MIXED -SI  RA  TEG  Y  SOLUTION 

Whenever  a  mathematical  or  formal  model  of  a  real  process,  in¬ 
cluding  a  conflict  situation,  is  formulated  significant  factors  are  ignored. 
Indeed,  the  modeling  process  consists  exactly  in  deciding  which  of  the  in¬ 
finite  variety  of  real  experience  must  be  ignored,  what  small  portion  must 
be  considered,  and  (often  the  easiest  part  of  the  modeling  operation)  how 
to  incorporate  the  effect  of  those  factors  which  must  be  considered.  There¬ 
fore,  a  fonnal  model,  which  is  intended  to  produce  recommendations, 
should  not  be  judged  a  failure  merely  because  the  recommendations  it  pro¬ 
duces  are  ambiguous  and  indefinite.  On  the  contrary,  such  ambiguity  - 
where  the  model  recommends  one  of  a  certain  set  of  actions  -  provides 
the  opportunity  for  incorporation  into  the  actual  decisions  of  the  many 
factors  ignored  in  the  model.  From  this  viewpoint,  a  solution  which  turns 
out  to  be  a  mixed  strategy  represents  three  separate  recommendations: 
first,  that  certain  pure  strategies  should  not  be  used;  second,  that  the 
complementary  set,  of  "good'1  pure  strategies,  may  sometimes  be  used; 
and,  third,  that  only  certain  probability-mixtures  of  the  "good"  pure 
strategies  should  be  used.  We  shall  discuss  each  of  those  recommenda¬ 
tions  in  turn. 

a.  Avoidance  of  Certain  Strategies 

The  implementation  of  the  first  portion  of  the  recommendation, 
that  certain  strategies  should  not  be  used,  is  extremely  easy  in  practice 
and  very  compelling  in  principle;  there  can  be  very  little  reason  for  ever 
choosing  any  one  of  those  pure  strategies  which  does  not  appear  in  any 
optimal  (minimax)  strategy.  Any  one  of  those  "non-good"  strategies  would, 
if  played,  provide  the  opponent  with  the  opportunity  to  gain  more  than  "his 
fair  share"  -  that  is,  more  than  he  could  obtain  if  both  sides  played  the 
minimax  strategies.  In  case  a  player  has  the  strong  conviction  that  his 
opponent  is  not  actually  playing  a  minimax  strategy,  he  might  then  wish  to 
take  advantage  of  this  fact  by  himself  deviating  from  die  set  of  good 
strategies.  We  will  refer  again  to  this  possibility,  but  for  the  present  need 
only  remark  than  an  opponent  who  appears  to  be  deviating  from  the  mini¬ 
max  strategy  is  either  playing  badly  or  else  he  is  playing  a  different  game. 
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(In  particular,  he  may  be  playing  a  non-zero-sum  game).  In  this  paper, 
we  are  restricting  ourselves  to  the  zero-sum  case.  Note  that  the  phrase 
"th.  opponent  appears  to  deviate  from  the  good  strategy"  suggests  observa- 
tion  of  the  opponent’s  strategy  over  a  number  of  repetitions  of  the  game; 
it  is  clear  that  much  game -theoretic  argument  is  implicitly  concerned  with 
adoption  of  policies  which  could  hold  not  only  for  an  isolated  play  of  the 
game  but  also  for  repeated  sequences  of  identical  games.  In  fact,  a  fre- 
}uentist  view  of  probability  would  require  that  the  notion  of  mixed  strategy 
be  bound  together  with  the  notion  of  repeated  games  or  repeated  opportun¬ 
ities  to  play  the  same  game.  From  a  subjectivist  or  a  synthetic  view  of 
probability,  however,  there  need  be  no  inconsistency  in  applying  the  notion 
of  mathematical  expectation,  even  though  the  game  is  assumed  to  be 
played  only  once. 

b.  Use  of  Certain  Strategies 

Now  we  turn  to  the  second  point,  that  determination  of  the  set  of  all 
mixed-strategy  solutions  specifies  which  pure  strategies  may  ever  be  cho¬ 
sen,  with  non-zero  probability,  in  a  minimax  solution.  It  is  one  of  the 
relementary  consequences  of  the  minimax  theorem  that  any  such  pure  strat¬ 
egy  will  provide  the  value  of  the  game  against  any  optimal  strategy  of  the 
opponent.  Therefore,  if  we  are  sure  that  our  opponent  will  be  playing  his 
minimax  strategy  against  us  (or  one  of  his  minimax  strategies  -  since 
uniqueness  is  not  guaranteed),  we  may  play  any  of  the  pure  strategies  which 
appear  in  any  of  our  optimal  mixed  strategies.  Within  our  model  of  the 
zero-sum  two-person  game,  it  is  assumed  that  both  players  have  exactly 
the  same  view  as  to  what  the  game  is;  that  assumption  implies  that  there 
is  no  uncertainty  in  rules,  outcomes,  or  payoffs  except  uncertainties 
which  are  exactly  described  by  probabilities  -  in  which  case  the  proba¬ 
bilities  must  be  known  to  both  players.  Consequently,  it  is  only  outside 
the  model,  and  within  the  context  of  the  real  applications,  that  we  can 
speak  of  things  being  known  to  one  player  and  not  the  other.  Of  course,  in 
real  applications,  it  is  not  always  clear  that  the  two  antagonists  have  exactly 
the  same  view  of  the  game  which  is  being  played;  judgment  must  be  em¬ 
ployed  in  assessing  fhe  recommendations  made  in  this  paper  insofar  as  they 
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rest  logically  upon  such  conditions.  To  summarize  this  second  point,  if 
there  is  one  of  our  pure  strategies  which  appears  with  a  non -zero  proba¬ 
bility  in  some  optimal  mixed  strategy  and  which  is  for  some  reason  more 
desirable  to  us  than  any  other  such  pure  strategy,  and  if  we  are  willing  to 
assume  that  the  factor  which  makes  it  more  desirable  is  absolutely  un¬ 
known  and  unsuspected  by  our  opponent,  then  we  may  with  impunity  play 
that  pure  strategy  instead  of  the  mixed  strategy  which  the  minimax  theory 
would  recommend. 

c.  Appropriate  Probabilities 

Addressing  the  third  jsoint,  the  relative  probabilites  with  which  we 
may  play  those  pure  strategies  which  do  appear  in  some  optimal  mixed 
strategy  for  our  side  -  we  have  already  discussed  the  question  of  when  we 
could  reasonably  choose  {with  probability  =  1)  one  of  those  pure  strategies. 

In  a  single  play  of  the  game,  no  meaningful  inference  can  be  drawn  from  the 
way  in  which  an  opponent  has  actually  played,  as  to  whether  his  strategy 
was  pure  or  mixed.  Along  the  lines  of  the  previous  discussion,  onr  may 
possibly  derive  information  by  considering  which  policies  would  be  rational 
for  an  cpponent  to  adopt  and  which  would  not.  If  there  is,  for  example, 
one  strategy  for  our  opponent  which  gives  him  the  opportunity  for  a  large 
gain  and  also  the  opportunity  for  a  large  loss,  and  which  would  have  simply 
been  described  in  advance  of  the  discovery  of  the  minimax  solution  as  "a 
case  for  the  psychologist"  or  a  case  where  thf  knowledge  of  one's  opponent  or 
intuition  were  required,  the  minimax  theorem  provides  a  quite  specific 
recommendation  to  either  avoid  that  strategy  or  to  play  it  or  possibly  to  play 
it  with  a  certain  probability  only.  The  basis  for  this  recommendation  is 
essentially  metatheoretic  in  nature.  Tb*t  is  to  say,  if  one  is  seeking  a 
rational  policy  and  if  the  rational  policy  is  expected  to  provide  explicit  re¬ 
commendations  of  what  one  should  do,  then  one  must  assume  that  the  opponent 
also  knows  of  the  existence  of  this  method  for  finding  rational  policies,  and 
would  be  able  to  apply  it  to  the  rules  of  the  game . 
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On  the  other  hand,  if  we  beMeve  .hat  our  pv  ice  pH  on  of  some  part  of  the 
rules  of  the  game  (either  of  tt  *  available  allcrr  "(P.vps,  01  of  the  payoffs, 
or  of  probabilities  attached  to  oh;.  noves;  is  different  from  the  cor¬ 
responding  perception  of  cu  oppoi  .it,  we  may  attivpt  to  "out¬ 
guess"  the  opponent.  For  example,  if  v  >eliev*  tha.  ht,  perceive.s  the 
game  as  a  zero-sum  tu  pipy  gan**  w*th  precise;/  deiii.ei  rules 
known  both  to  us  and  to  him,  n  >e  ■  .r.g  to  as  sum*  that  he  is 
playing  a  strategy  whic  h  wou  /  ,  in  T  .  It  w  perceive  the  . 

game  as  being  a  somewhat  t’itf  ^ent  gaitie,  In  ,  b"f  nevertheless 

are  willing  to  assume  that  u.c  ,  1/  .r  r  v  ^  the  game  as 

r*  has  not  occurred  to  oiu  oppoi>  .) . %  »■  de  to  choose, 

that  strategy  which  will' provide  tj  ealest  t. <.  yoli  to  us  in  T' under  the 
assumption  that  our  opponent  will  be  chocsing  one  of  the  mixed  strategies 
which  is  optimal  for  him  in  T  .  If  he  has  only  one  such  optimal  strategy, 
,  our  choice  is  then  clear;  we  choose  the  strategy  which  optimizes 

against  ij  .  If,  however,  17  is  not  unique,  it  will  generally  happen  that 
the  payoffs  resulting  for  us  in  I”  will  depend  both  on  which  optimal  t) 
was  chosen  by  the  opponent,  and  on  which  strategy  we  uhoose.  If  we  then 
attempt  to  optimize  within  that  set,  it  is  hard  to  find  a  logical  process 
which  will  permit  us  to  take  full  advantage  of  our  superior  knowledge  (we 
know  the  game  is  I”  ,  while  he  thinks  it  is  T  ).  We  should,  under 
those  assumptions,  probably  treat  the  opponent's  choice  of  an  ^  from 
within  that  set  of  optimal  strategies  as  a  purposeless  choice,  and  employ 
one  of  the  conventional  solutions  to  "games  against  Nature". 


IV.  "OUTGUESSING11 

Because  there  are  significant  differences  between  the  min-max  in 
pure  strategies  and  the  max-min  in  pure  strategies  ,  and  because  the  actual 
value  of  the  g  ,.ne  generally  lies  strictly  between  those  limits ,  we  would  be 
in  effect  surrendering  to  the  opponent  an  advantage  if  he  did  in  fact  perceive 
the  possibility  of  our  interest  in  F'  .  Although  this  discussion  has  men¬ 
tioned  what  one  player  knows ,  what  one  player  wants ,  what  one  player  thinks 
the  other  player  knows ,  etc .  ,  it  should  be  noted  that  all  these  statements 
of  knowledge  can  easily  be  made  quite  formal  and  rigorous.  One  especially 
important  situation  can  occur  when  each  opponent  has  a  certain  set  of  opin¬ 
ions  about  the  desires  of  the  other,  and  subjective  probabilities  as  to  which 
of  those  opinions  is  true .  If  the  game  is  truly  zero-sum,  and  if  it  is  only 
to  be  played  once ,  and  it  the  players  share  a  common  subjective  probability 
on  the  various  alternatives ,  then  the  payoffs  may  be  averaged,  and  the 
•situation  may  be  subsumed  in  the  ordinary  zero-sum  case.  That  is  to  say, 
one  can  formally  demonstrate  that  the  minimax  mixed  strategies  are  the 
unique  reasonable  way  in  which  a  play  should  behave.  This  remains  true 
if  the  game  is  repeated,  provided  that  each  player's  state  of  information, 
about  the  other's  true  desires,  does  no’,  change;  consequently,  the  repeat¬ 
ed  zero-sum  game  with  some  uncertainties  about  the  rules  and  payoffs 
does  not  in  fact  present  a  new  problem. 

There  are, however ,  two  directions  in  which  this  model  can  be  ex¬ 
panded,  in  order  to  encompass  a  greater  variety  of  realistic  situations. 
First,  the  model  may  be  extended  to  include  non-zero -sum  games;  second, 
we  may  assume  that  players  may  learn  from  the  earlier  plays  of  the  game 
some  facts  which  may  influence  their  behavior  on  later  plays.  Efforts  to 
combine  these  two  extensions  have  so  far  been  unsuccessful. 

In  the  non-zerc-sum  case  ,  one  may  ask  either  "What  are  :  ason- 
able  ways  for  the  players  to  behave,  if  they  are  not  allowed  to  negotiate?" 
or  ore  may  ask  "What  are  reasonable  ways  for  the  players  to  negotiate?" 

We  may  note  that  the  use  of  subjective  probabilities,  as  described  above, 
may  load  to  a  non-zero-sum  case  if  the  two  players  do  not  use  the  same 
subjective  probabilities. 
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It  appears  that  competitive  behavior,  with  no  vestige  of  negotia¬ 
tion,  must  result  in  an  equilibrium  point  if  there  is  only  one  such.  If 
there  are  several  equilibrium  points,  the  players  may  lose  by  being 
unable  to  cooperate  on  the  same  one.  The  "Prisoners'  Dilemma"  paradox 
shows  that  thi.1'  situation,  even  when  the  results  are  unambiguous,  is  often 
quite  unsatisfactory. 

If  we  allow  the  players  to  negotiate,  there  are  various  assumptions 
possiole  about  the  permitted  types  of  bargaining  and  negotiation.  Several 
specific  problems  of  this  type  have  been  handled  by  Harsanyi,  Selten,  and 
others,  who  have  formalized  this  notion  of  "bargaining  game",  and  proven 
theorems  about  existence  and  uniqueness,  in  case  each  side  merely  makes 
a  single  proposal  (so  that  "cooperation"  occurs  if  the  two  proposals  are 
consistent),  and  also  in  case  a  sequence  of  offers  and  counter-offers  may 
be  made  (so  that  the  notion  of  "cooperation"  must  be  interpreted  much 
more  broadly).  This  last  model  is  extermely  complicated,  and  the  solu¬ 
tion  of  significant  special  cases  has  not  yet  been  accomplished. 

If  the  repetitive  zero-sum  model  is  extended  in  the  second  direc¬ 
tion,  so  as  to  encompass  the  situation  where  each  player  learns  something 
from  the  behavior  of  his  opponent  in  earlier  stages  of  the  game ,  then  it 
may  happen,  as  plays  of  the  game  are  repeated,  that  both  parties  effective¬ 
ly  accumulate  total  information  as  to  the  actual  payoff;  or  alternatively  it 
may  happen  that,  even  after  a  large  number  of  plays,  residual  uncertainty 
remains.  The  "limiting"  game  appears  similar  to  the  basic  case  where 
the  uncertainty,  and  the  probabilities,  are  known  to  both  players. 

On  the  other  hand,  some  heuristic  notions  can  be  formulated  which 
provide  useful  guidance  in  case  an  opponent  appears  (in  a  sequence  of  plays 
of  a  game  T  )  to  be  playing  badly.  Three  explanations  should  be  considered: 


one,  he  is  indeed  stupid;  two,  he  is  intelligently  playing  some  other  g'».r*'e 
T  *  !  or  three,  he  is  trying  to  "mousetrap  <«s  by  pretending  '~n«> 
first  tv,”  -  j  fc.'  ucnate  irom  the  set  of  our’optimal  strategies 

and  then  taking  advantage  of  the  deviation.  Consideration  of  the  second 
case  is  very  difficult  unless  v/e  can  formulate  explicit  notions  of  reason¬ 
able  possibilities  for  J*  *  ,  but  a  response  to  both  the  first  and  third  can 
be  combined  under  this  policy: 

choose  our  strategy  so  as  to  optimize  our  expected  payoff 
against  his  observed  historical  average  strategy,  subject 
to  the  restriction  that  we  do  not  expose  ourselves  to  a 
larger  expected  loss  on  any  one  play  than  we  have  gained 
through  his  non-optimal  play  in  the  past. 


•  IV- 11 


CHAPTER  V 


REJECTION  OF  DATA:  OPTIMAL  ESTIMATION 

t 

IN  COMPLICATED  DISTRIBUTIONS 


John  P.  Mayberry 


V-l 


D 

B 

I 

I 

! 

I 

I 

I 

D 

D 

D 

U 

D 

II 
fl 
I 
I 


1 .  INTRODUCTION;  THE  BASIC  CASE 

In  this  paper  we  shall  consider  several  problems  of  estimation 
of  parameters  in  statistical  distributions.  We  intend  these  problems  to 
be  useful  and  reasonable  generalizations  of  the  more-usual  case  of 
independent  samp!/  s  from  a  normal  (Gaussian)  distribution. 

Let  us  i  ecall  the  solution  to  that  common  case  .  If  we  have  a 
sample  X  =  {xj  of  N  real  number  .1 ,  x.  ,  x_  ,  . . .  ,  x  ,  which  are 

1  X  c*  IN 

assumed  to  be  independent  observations  from  a  Gaussian  population  of 

2 

unknown  mean  \i  and  variance  Or  ,  then  the  MLE  (maximum- likelihood 

2 

estimates) of  /j,  and  a  are 


(1.1) 


—  ^Xi 

#MUC  S  x  =  -N1  ’ 

,  £  (x.-x)2  Ex2 

tsA  .  i  i  — i 

"mue  '  5  ”  TT  -  x 


It  turns  out  that  the  modification 


(1.2) 


£  (x.  -  x)2 

N-l 


has  a  smaller  bias  . 

2.  EXAMPLE:  MIXTURE  OF  TWO  NORMAL  DISTRIBUTIONS 

Our  first  substantive  example  concerns  independent  samples  from 
a  distribution  which  is  believed  to  be  a  mixture  of  two  normal  distribu¬ 
tions,  with  a  common  but  unknown  mean  fji  and  different  standard 
2  2 

deviations  ^  and  0^  (both  also  assumed  unknown).  We  may  assume 
without  loss  of  generality  that  Oj  <  cr^  ,  and  denote  by  p  the  probability 
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2  2 

that  each  one  of  the  measurements  comes  from  the  first  (or  =  ) 

distribution.  Then  the  likelihood -function  for  a  single  measurement  x  is: 


(2.1) 


X\  (x  I  ai ’  °z,tXl  p> 


o~  jm  • nexp 


Tfr  •  nexp 


(£>)' 


where  nexp  (x)  denotes  exp  (-x)  =  e 

Although  there  are  probably  no  difficulties  of  principle  in  estimating 
these  four  parameters  from  a  sample^of  moderate  size,  we  can  reasonably 
consider  the  parameters  p  and  k  =  ~~>  1  as  being  known  in  advance, 
since  that  facilitates  the  derivation  of  approximate  solutions  to  the  MLE 
equations.  Furthermore,  the  procedures  we  shall  derive  below  are  not 
very  sensitive  to  the  exact  values  assumed  for  p  and  k.  With  that 
assumption,  the  above  expression  (2.1)  simplifies*  to 


(2.2) 


X  (x  |  a,  pi)  = 


fa  ( p  • nexp  iff-) 1  ^  •  ne*p(sy) 


Now,  if  we  have  our  set  %>=  {x.}  of  W  independent  samples  from 
that  distribution,  we  get  the  joint  likelihood  function  in  the  form 


*  We  have  also  written  a  for  cr ^  . 
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(%l 


M)  = 


(2.3) 


(27T) 


2  '  a'N  ']T  (p  •  nexp(<7J_)  +i?  '  nexp(^77i) 


Now  we  may  develop  the  MLE  by  solving  the  equations 


~  X3  (X  |  a,  //.)  =  0  , 


(2.4) 


aF  ^  <x  i  °* 


M)  “  0  • 


After  some  tedious  algebra ,  we  reach  a  conclusion  which  can  be  easily 
expressed  with  the  aid  of  some  new  notation.  Let 


(2 . .») 


X  =  k 


-2 


\  =  p  •  nexP  l  Tc* 


)  • 


B.  =  (‘”£)  .  nexp 

A.  +  B.  .X 

w  =  _i - - - 

i  A.  +  B. 


£8)  • 


(Note  that  A^,  B. ,  and  all  depend  on  fi  and  on  O  ;  we  do  not  make 
that  dependence  explicit,  in  order  to  keep  the  notation  simple.) 
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The  maximum -likelihood  estimates  of  /x  and  or  can  now  be  expressed 
in  terms  of  the  W.  ,  recognizing  that  the  W  will  in  turn  depend  on  the 
values  assumed  or  computed  for  JX  and  O  .  We  have  confidence  that  the 
procedure ,  if  iterated  a  modest  number  of  times  by  recalculation  of  the 
and  the  /X  and  O  ,  will  generally  converge .  We  also  develop  below  a 
useful  crude  approximation  which  can  be  used  for  an  initial  estimate  of 
the  W  . 

Using  the  notation  of  (2.5),  the  maximum -likelihood  estimates 
can  now  be  expressed  as  follows: 


MLE 


L.  W.  x. 
1  1 _ l 

a  w. 

i  i 


(2.6) 


~  2 
a 

MLE 


n  h  wi  (v">2 


1  v  w  ,  2  2 
N  Si  Wi  <Xi  "  1  • 


(As  a  consequence  of  the  first  equation  for  fi  ,  the  two  expressions  for 
2 

O  given  in  (2 .6)  can  be  shown  to  be  equivalent. )  Just  as  in  the  prototype 

case  cf  the  Gaussian  distribution,  bias  in  the  estimate  is  reduced  by  employing 

2 

an  alternate  estimator  fv/?  cr^  ,  viz. 


(2.7) 


E.  W.  (x.  -  £)2 

N  _  i 


rather  than  those  of  equations  (2.6). 
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Now,  we  can  interpret  the  W.  as  a  more  precise  answer  to  the 

frequent  question  of  "when  shall  we  reject  an  outlier,  or  an  apparently 

wild  observation?"  (Of  course,  we  have  made  assumptions  which  are 

more  narrow  than  the  usual  ones ,  so  it  is  very  natural  that  our  answers 

are  more  precise;  but  the  relationship  is  surprisingly  close  .)  Recall  that 

-2 

X  ,  which  is  equal  to  k  ,  is  a  number  smaller  than  1  -  generally  much 

2  2  2 

smaller,  because  the  variance  of  k  O'.  =  CT_  is  intended  to  explain 

i  w 

observations  which  would  be  very  improbable  in  a  Gaussian  distribution 
2 

with. variance  .  Then,  recalling  that  A.  and  B.  are  strictly 

positive ,  the  last  equation  of  (2.5)  implies  that  W.  will  be  always 
strictly  between  X  and  1  ;  when  A^  «  X  ,  W^  is  nearly  equal  to  X  . 
We  may  approximate  W^  by  1  whenever  A.  >  B.  ,  and  by  X  whenever 
A.  <  Bj  X  .  Recalling  the  definitions  given  in  (2.5)  for  A.  and  B.  ,  we  can 
deduce  (after  some  algebra)  a  crude  approximation  to  W^  ,  as  follows: 


i^i  <  .*,*>> 


(2.8)  whenever  / 


1^1  ”  .  w 


V 


We  have  prepared  a  small  table  of  those  expressions  ,  which  might 
be  called  the  "accept"  and  "reject"  limits  respectively;  since  X  «  1  ,  to 
include  an  observation  with  weight  W^  =  X  is  nearly  the  same  as  to  exclude 
it  altogether  .  Between  those  limits  ,  which  are  expressed  (like  (2  .8)  )  in 
terms  of  |  |  ,  there  will  be  a  "partial  acceptance"  -  in  other  words, 

a  value  of  W.  intermediate  between  1  and  X  . 

The  procedure  which  results  from  this  set  of  hypotheses  is  not 
very  different  from  the  rules  of  thumb  often  used  for  rejecting  "outliers"; 
the  logical  foundation  for  those  results  is  of  course  much  clearer. 
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TABLE  I:  Critical  Values  of  |  |.  for  Various 

Combinations  of  p  and  k  . 


k 

"Accept" 

"Reject" 

0.9 

2 

2.5 

2.9 

0.9 

10 

3.0 

4.3 

0.9 

100 

3.6 

5.1 

0.9 

1000 

4.3 

6.0 

0.99 

2 

3.3 

3.7 

0.99 

10 

3.8 

4.8 

0.99 

100 

4.3 

5.9 

0.99 

1000 

4.8 

7.1 

3.  EXAMPLE:  MIXTURE  OF  SEVERAL  NORMAL  DISTRIBUTIONS 

We  extend  the  results  of  Section  2  above  to  the  problem  of  estimating 

the  smallest  (and  most  probable)  of  several  standard  deviations.  We  have 

m  distributions  ,  all  with  a  common  mean  fi  ,  with  standard  deviations 

and  probability  of  occurence  ,  for  j  =  1 ,  2  ,  . . . ,  xn  . 

We  define*  k.  =  (CT.  /  a, )  ,  for  j  =  1 ,  2  ,  and  assume  the 

J  J  1 

k.  and  the  p.  known  a  priori .  We  shall  use  0  as  a  Synonym  for  0^  , 

J  J 

from  now  on . 

Then  (2.2)  becomes 


*  Of  course , 


k. 


=  1  always  in  this  notation. 


and  the  analog  oi  (2.3),  when  we  have  observed  a  set  X  =  {x.  |  i  -  1,2,...,] 
o i  N  independent  samples,  is 


(X|or,M)  = 


(3.2) 


N 

‘2 


N  m 


(27 n  -  .  Cr"N  .  j[  ^ 

i=l  j=l 


'(x.-Mr\ 

2  2  ) 
1.0/ 


nexp 


v2k 


Sl 

k. 

3 


Now  again  we  wish  to  develop  MLE  ,  so  we  set 

( 


(3.3) 


dor 


log  (fi  (X|a,jU)  =  0  , 


< 


log  ^  (X|?,M)  =  0  . 


Explicitly,  we  can  express  the  first  of  those  two  results  as 


(3.4) 


0  = 


which  is  equivalent  to 
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Now  we  note  that 


(3.10) 


W.  = 


a.,  +r.m,  (a..  .  k:2) 

il  j=2  u  J 


1  A  +  E.m-  A.. 

il  j=2  ij 


It' we  compare  this  with  equations  (2.5),  we  see  that -we  could  make  the 
computations  here  as  simple  as  those  of  (2.5)  if  we  defined  A^  =  A.^  , 


(3.11) 

A. 

l 

=  Ail  * 

(3.12) 

B. 

l 

vm 
*  Sj=2 

.  £m_ 

(3.13) 

-  _izL. 

IJ 


B. 

l 


(Th‘.  only  real  complexity  is  that  X^  depends  on  i  in  this  case.) 


"n  fact,  each  Xj  is  a  weighted  average  of  the  k^  .  To  get  a  crude 
procedure  correspor  ding  to  the  procedure  (2  .8),  we  may  simply  drop  the 
exponential  term  of  (3.7),  and  define 


(3.14) 


X  = 


Si*2  (p.i  '  i  * 


<»i  •  k;‘> 


(3.15) 


k=W  = 


-e™2  <Pj  •  k;3»- 


v-io 


1 

.2 


and 

(3.16)  P  =  Pj  . 

and  use  the  critical  values  developed  as  in  Table  I,  above ,  to  determine 
initial  weights  and  "rejection  levels". 


4.  Bias  Affecting  Several  Observations 

More  complex  than  the  cases  considered  above  is  the  ca3e  where 

♦ 

a  common  effect  may  bias  several  measurements  .  We  treat  first  the  case 

where  an  m-by-n  matrix  of  elementary  measurements  is  taken  We 

assume  that  each  of  the  m  rows  consists  of  a  set  of  n  independent 

2 

samples  from  a  normal  distribution  with  variance  a  and  a  common  mean; 
with  probability  p  that  row  mean  is  fx  ,  and  with  probability  (1-p)  it  is 
fX1  £  fl  .  (This  is  intended  to  be  an  approximation  to  the  result  of  an  anomaly 
in  sonar  transmission  paths  -  some  of  our  sets  (rows)  of  measurements  are 
estimating  the  correct  quantity ,  some  are  estimating  something  else.) 

Then  the  probability  of  a  result  X„  as  the  j'th  measurement  in  the 
i'th  row  is 


(4.1) 


jU.  ,  the  mean  in  the  i'th  row,  is  either  fx  (which  occurs  with  prob- 
p  )  or  fx’  (which  occurs  with  probability  (1-p)). 

The  likelihood  of  a  set 

(4.2)  X~  (X..} 


where 

ability 


of  sample  values  is  then  given  by 


*V.«  *■  /-’'tfWiTWl  )*■> 


(4.3) 


ofjj  (Xk,M.M’fp)  = 


F  (pF  ^  nexp(¥L) 

1=1  '  j=i 

JL  /(X..-M’f\ 

(l-p)Tf  -7=:  nexpf— ^ - ) 

J  ,  ffv^Tr  •  \  2a  J 

J=1 


(27T)  2  a"mn  TT  [  p  nexpl 


/  /r  (x  -m)  x 

p  nexpl  -* - * -  I 

\  \  Z  or2  / 


/Si  <Xii  ~ 

+  (1-p)  nexpl  - :* - 


Neglecting  the  constant  first  factor,  we  will  get  the  Maximum- Likelihood 
Estimates  by  maximizing,  instead  of  ,  the  quantity 


(4.4)  <  =  ff  ♦ 


Assuming  p  known  in  advance,  we  must  set 


(1-p)  .  nexp 


E.(X„-ji') 


m 

\  20 


V-l  3 


fi 


(4.7) 


Define 


(4.8) 


Then  equations  (4 . 
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- - — 


flFSP 


» »v~4  imwwww  cn.»  wyftTiKeiocrrri  ■ 


I 

I 

I 

i 


a 

D 

0 

Q. 

0 

D 

0 

D 

0 

D 


(4.9)  < 


>m 


t  t 


a  = 


mn 


H  = 


2.  R.  X, 

l  l  i. 

£.  R{ 


M  = 


£i  Ri  xi. 

EiRi 


£.  X.. 
X,  =-L-U- 
i.  n 


«  and. 


,  where 


This  algebra  has  a  natural  intuitive  appeal;  R.  is  the  posterior  probability 
that  the  measurements  of  the  i'th  set  have  come  from  the  distribution  with 
mean  [i  ,  and  (1-R^)  is  the  posterior  probability  that  those  of  the  i’th 
set  have  come  from  the  other  distribution. 

It  is  important  to  note  that  these  equations  are  necessary,  but  not 
sufficient;  it  is  quite  possible  that  alternative  choices  could  give  local 
maxima  of  the  likelihood  function. 
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A  SIMPLE  I-GAME 

As  a  case  study  of  the  application  of  I-game  methodology  to  ASW 
detection  problem  we  will  set  up  a  scenario  involving  two  alternative 
missions  for  the  Red  submarines.  When  it  is  assumed  that  Blue  does 
not  have  prior  information  as  to  which  of  the  missions  Red  is  assigned, 
the  conflict  can  be  modeled  as  an  I-game  by  having  Blue  act  as  a 
Bayesian.  This  means  that  he  must  attach  prior  probabilities  p  and 
1-p  to  the  two  missions  (which  are  assumed  to  affect  the  game  payoffs 
substantially)  and  then  optimize  his  expected  payoffs.  It  has  been  shown 
[;l]  that  this  is  equivalent  to  a  game  in  which  a  unique  "chance"  move 
first  determines  the  payoffs  according  to  the  same  mixture  (p,  1-p);  then 
the  Bayesian  players  choose  their  strategies.  One  or  both  may  be  ignor¬ 
ant  of  the  outcome  of  the  chance  move.  We  assume  that  only  Blue  is 
ignorant. 

Our  scenario  follows.  Red  will  attempt  to  transit  a  large  rectangle 
of  ocean  with  T  hours  under  one  of  two  alternative  game  conditions :  (i) 
The  mission  is  essentially  reconnaissance  and  a  Red  submarine  must  try 
to  obtain  information  and  return  to  base  undetected.  If  the  Red  submarine 
is  detected,  its  information  becomes  worthless,  (ii)  The  mission  requires 
that  one  of  several  Red  submarines  must  come  within  range 

of  a  Blue  shore  target,  preferably  undetected.  As  many  as  possible  of 
the  Red  submarines  should  evade  detection  in  order  to  maximize  the 
benefits  of  the  mission.  We  now  describe  two  simple  strategies  for 
each  side. 
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Red1!,  physical  strategies:  R^  =  Send  one  or  two  submarines  and 

take  evasive  actions  for  maximum 
security. 

.  R^  =  Send  several  submarines,  some  of 

which  are  intended  to  "blast  through" 
even  at  considerable  risk  of  being 
detected. 

Blue's  physical  strategies:  =  Deploy  ASW  forces  in  the  area  so  as 

to  obtain  maximal  pursui1.  capability 
when  a  possible  target  signal  is 
.  received. 

B  =  Defensive  deployment  which  is  best 
£ * 

against  R^ 

Now  there  are  two  games  which  we  will  label  G^  and  G^.  Suppose  the  payoffs 
(as  measured  in  terms  of  a  weighted  sum  of  detection  and  counterdetection 
probabilities)  are  as  follows:  •  .  - 


The  values  of  these  two  games  are:  v(Gj)  =  3/5  and  v(G2)  =  -  1/3. 

Optimal  mixed  strategies  are:  (l/5Rj,  4/ 5R^)  and  (4/5B^,  l/SB^forGj 

and  (  l/SRj,  2/3R2)  and  (  l/3Bj,  2/3B2)  for  G2 

Red  knows,  which  game  he  must  play;  Blue  must  guess.  If  Blue  has  a  priori 
probability  p  for  Gj  being  the  true  state  of  the  world,  and  of  course  1-p 
for  G2,  and  if  he  cannot  a^oid  letting  Red  know  the  value  of  p,  the  Bayesian 
I-game  model  suggests  the  following  reduction:  both  might  play  a  new  zero- 
sum  gamed(p)=  pGj  +  (3 -p)G2  which  has  payoffs  as  follows: 


3p/2-l 


The  value  ofCj(p)  is  v*  =  and’ the  optimal  mixed  strategies  are 


i  ^  "p/^p  _JL__r 

l3-p/2rl  *  3-p/2K2 


R?)  and  1  ’  JIp/2 — ^2).  ca^  seen  by 


substitution,  when  p  =  0  the  gameG(p)becomes  G 2,  and  when  p  =  1,  it 
becomes  G^;  in  both  cases  the  value' and  optimal  strategies  are  the  same 
as  given  above.  However,  in  the  absence  of  any  prior  information,  Blue 
would  have  to  play  indifferently,  which  means:  p  =  l/Z.  The  resulting 
value  and  optimal  strategies  are:  l/ll,  (  3/lIR^,  8/llR2)  and  (  6/llB^, 
5/llB2).  If  we  assume  that  Red  also  plays  as  a  Bayesian  with  prior  prob- 


VI-4  • 


ability  p  =  l/2,  and  that  he  does  not  know  whether  or  is  actually  in 
force,  then  these  are  the  correct  quantities  for  an  optimal  solution  of  the 
I-game.  However,  when  we  allow  Red  to  use  his  information  and  differ¬ 
entiate  his  strategies  according  as  G^  or  G^  is  in  force,  we  will  show  that 
he  can  do  better.  The  precise  interpretation  of  the  policy  is  a  matter  for 
careful  study  by  the  operational  command,  but  one  notices  that,  however 
the  mixed  strategies  may  be  interpreted,  the  range  of  the  weights  attached 
to  Blue's  first  strategy,  B^,  is  quite  large:  4/5,  6/11,  1/ 3  for  p  =  1,  1/ 2 , 

0  respectively.  Inasmuch  as  B^  and  B^  involve  substantially  different  de¬ 
ployments  of  the  ASVV  forces,  the  three  policies  will  most  likely  result  in 
noticeably  different  actions. 

The  fact  that  Red  knows  which  of  G^  and  G^  is  being  played,  and 
Blue  does  not,  may  be  represented  in  a  4  x  2  matrix  game,  G(p}.The  pure 
strategy  choices  for  Red  are  expanded  to  four  by  allowing  him  to  act  dif¬ 
ferently  according  as  the  game  is  Gj  or  G^.  The  same  option  is  not  avail¬ 
able  to  Blue. 


Admittedly  a  strange  assumption  in  our  example;  we  treat -this 
case  only  for  the  sake  of  completeness  of  the  discussion  of  the  game- 
theoretic  method. 
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The  solution  of  G  (p)  is  as  follows: 

•  « 

*  • 

If  o  4  p  £  4/5,  v  (p)  -  (-2  +  7p)/ 6,  and 

Red',  optimal  strategy  is  [(o)  Ry,  (o)  R12,  (5 ^-)  Ry,  R22]' 

Blue’s  optimal  strategy  is  Ql/3)  B^,  (2/3)  Bj. 

If  4/5  ^  p  £  1,  v  (p)  »  3/5,  and 
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The  upper  graph  in  Figure  1  is  the  value  of  the  game  G(p);  it  is  always 
higher  than  either  v*(p)  or  the  "reveal"  line  ,  which  represents  the 
value  if  Red  were  to  announce  to  Blue  which  game  was  the  true  game 
after  "chance"  had  chosen  or  G^  with  probabilities  p  and  1-p. 
Thus  we  see  that  Red  can  take  advantage  of  his  superior  information  to 
gain  a  higher  payoff  b.y  playing  the  I-game  as  if  it  were  G(p).  Only  when 
p  =  0  or  1  does  he  not  make  a  positive  gain  compared  with  the  other 
methods  of  optimizing.  It  is  also  worth  noting  that  when  p  <4/5,  Red 
cannot  achieve  the  "best"  result  of  v  =  3/5.  A  naive  interpretation  of 
the  I-game  might  lead  one  to  believe  that  Red  merely  has  to  play  optimal 
strategies  for  whichever  the  true  game  happens  to  be  to  achieve  this 
result.  But  this  ignores  the  ability  of  Blue  to  infer  what  the  true  game 
ie  from  Red’s  mdves  and  act  according  to  his  inference.  When  p 

exceeds  4/5,  however,  Red  can  successfully  play  to  the  full  advantage 

•  •  « 

of  his  superior  information.  x  '• 

A  thorough  analysis  of  the  repeated  I-game  would  show  that  Red 
can  only  improve  on  G*(p)  to  the  extent  of  the  "reveal"  line  mentioned 
above.  His  optimal  strategy  requires  that  he  should  (in  the  limit  as  the 
number  of  repetitions  becomes  indefinitely  large)  optimize  for  whichever 
of  G-  or  G,  is  the  true  game.  Blue  would  soon  discover  from  Red's 
moves  which  is  the  true  game,  but  Red  would  lose  more  if  he  attempted 
to  conceal  it  from  Blue.  In  this  report  we  have  treated  the  example  as 


1 

The  line  joining  (0,  -1/3)  and  (1,  3/5). 
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a  single  instance  of  the  I- game  and  thus  have  not  emphasized  the 
analysis  of  limit  strategies' for  repeated  games. 
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APPENDIX*  :  An  Alternative  Viewpoint 


To  round  out  the  paper  ,  let  us ,  in  fact,  look  at  G  =  (Gj  ,G2) 
from  Red's  point  of  view. 

We  repeat  the  assumptions. 

(i)  Blue  and  Red  know  p  (the  a  priori  probability  of  nature 
choosing  G^) 

(il)  Red  knows  "nature's  choice"  of  which  game  Gj ,  or  G^,  is 
being  played,  Blue  does  not. 

We  determine  the  value  of  the  game ,  given  Red  knows  which 
game  is  being  played.  Four  cases  arise: 


(1)  p  <  4/5 

Blue's  strategy  (identical  for  both  games,  of  course)  was  noted 


previously:  [(1  /3  Bj  ,  (2/3)  B^] 
Game  :  Gj 

Red's  Strategy  :  [(OjRj  (1)R2] 
Expected  Pay-off:  5/6 


G. 


r/L IE/*)  r  /Ll* 2/2  )r1 
3  -  3p  '  Rr  (3  -  3p  ,R2J 

-1/3 


(2)  p  >  4/5 

Blue's  strategy  (both  games):  [(4/5)  B^,  (1/5)  B2] 


Game 


G, 


Red's  Strategy  :  [(Sj?  jj'*)  »  (^")  R23  [(1 )  Rj  ,  (0)  R2] 


Expected  Pay-off :  3/5 


3/5 


*by  —  A.  J.  Truelove 
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Summarizing  Expected Pay-offs: 


/ 

GAME 

A  priori P  (Game) 

Expected  Pay  Offs 
p  <  4/5  p£  4/5 

P 

5/6 

3/5 

°2 

1  -  p 

-  )  /3 

3/5 

Given*  priori  probabilities  for  (Gj,  G2)  of  (p,  i-p),  Red's  expected 
pay-off,  before  Red  know  a  which  game  nature  has  chosen  (and  hen1, 
the  value  of  the  game  (G^  ,  G^)  given  p)  is  given  by: 


Expected  Pay-off  from 

(Ql»  g2) 

p  <  4/5 

(5/6)  p-  (1/3)  (1  -p) 

s  (-2  +  7p)/6 

P  > 

3/5 

Now  consider  the  game  G* ,  and  suppose  that  Red  is  constrained 
to  play  the  mixed  strategy: 

'3  -  p/2  Rl’  3  -  p/2  *Z’\ 

even  though  he  knows  which  of  Gj ,  G^  is  being  played.  This  situation 
would  arise  if  Red  did  not  wish  to  give  Blue  information  on  which 


game  is  being  played.  (Of  course,  the  pay-offs  would  have  to  be 
concealed  from  Blue  also . ) 

Blue's  strategy  is  B^ ,  |  "  and  the  value 

of  G*  (p)  is  v*  =  2  . 

r  6  -p 

As  noted,  Red  can  "reveal"  to  Blue  which  game  is  being  played, 
by  deviating  from  his  best  strategy  for  G*. 

We  now  calculate  the  value  of  G*,  given  nature's  actual  choice 
(Gj  or  G^)l  i.e.,  we  adopt  Red's  point  of  view. 

Using  the  strategies  for  G*  : 


Red 

Chooses 

Blue 

Chooses 

Probability  of  this 
pair  of  choices  X 
(3  -  p/2  f 

Pay-Off  if 
G> 

game  is: 

1 

1 

1  -  (1  /2]p  -  (1  /2]p2 

1 

1 

1 

2 

2  -  (5/2)p  +(3/4)p2 

-1 

-1 

2 

1 

2  +  2p 

1/2 

-1 

2 

2 

4  -  3p 

1 

0 

Expected  Pay-Off 

5p2-4p-l6 

5p2-4p+12 

(P  -  6)2 

(P  -  6)2 

We  verify  the  previous  result  that  the  overall  expected  pay-off 
(before  nature  has  chosen  G^or  G^ ,  i.e. ,  from  Blue's  point  of  view)  is: 

(p-6)‘Z  (5p2  -  4p  -  l6)p  +  (5p2  -  4p  +  12)  (1  -  p) 


(5p  -  2)/(6  -  p)  =  v* 


Thus ,  even  in  the  repeated  game  ,  Red  always  does  best  to 
reveal  the  game  being  played  (even  if  the  pay-offs  were  concealed 
from  Blue).  In  conclusion ,  we  note  that  this  is  not  always  true  (see 
T.  Saaty,  ’’Mathematical  Models  of  Anns  Control  and  Disarmament,” 
Wiley,  1968,  p.  124,  sec.  4.4;  based  on  Aumann  and  Maschler  ,  Game 
Theoretic  Aspects  of  Gradual  Disarmament  in  "Development  of  Utility 
Theory  for  Arms  Control  and  Disarmament,”  Contract  No.  ACDA/5T-80 
and  116  with  MATHEMATICA,  Princeton,  New  Jersey  (1966). 
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CHAPTER  VII 

THE  GENERALIZATION  AND  SOLUTION 
OF  THE  SMALL  I-GAME 
DETECTION  MODEL 


Francis  M.  Sand 


> 

VII- 1 


The  Generalization  and  Solution  of  the  Small 
I- game  Detection  Model. 

In  the  previous  paper, an  I -  game  formu- 
lation  of  ASW  barriers  and  penetration  strategies  was  presented  and  solved. 
The  payoffs  were  intended  to  represent  weighted  combinations  of  the'  v 

probability  of  detection  of  a  submarine  and  the  probability  of  counterdetection 
by  a  submarine.  The  differing  utilities  of  the  two  events  "detection"  and 
"counterdetection",  both  within  a  mission,  and  as  between  two  different 
missions,  were  not  taken  into  account.  This  paper  will  extend  the  scope  of 
applicability  of  the  method  by  treating  the  payoffs  as  expected  utilities.  The 
assumption  that  both  and  Gg  are  zero-sum  games  will  be  retained.  A 
further  assumption  will  be:  that  the  probabilities  of  detection  and  counter¬ 
detection  are  the  same  in  all  similar  strategic  situations.  In  other  words, 
these  probabilities  do  not  depend  a  priori  on  the  mission  type. 

The  framework  of  the  I- game  model,  using  the  two  assumptions  of 
the  foregoing  paragraph,  is  represented  below  for  the  same  two  games 
Gj  and  G^  described  in  the  previous  case,  allowing  now  for  different 
utilities  in  the  two  situations  however.  Note  that  we  do  not  allow  the  utilities 
to  be  affected  by  any  considerations  except  (i)  the  mission  type  or  game 
situation  (G^  or  G^)  and  (ii)  the  event  typedetection  or  counterdetection.  In 
the  present  working  paper,  no  interaction  between  the  event  type  and  the 
mission  type  or  between  the  time  sequence  of  events  and  the  mission  is 
considered  in  the  model. 


Let  u^  =  U  (counterdetection  of  a  Blue  ASW  vehicle  j  G..) 
v.  -  U  (detection  of  a  Red  submarine  G^) 
p„  =  Prob  [counterdetection |  strategy  choices  R.  and  B.] 
=  Prob  [detection  strategy  choices  R^  and 
with  i,  j  =  1 , 2  . 

In  the  game  matrices  which  follows,  the  payoffs  are  expressed  in 
utilities  gained  by  Red  who  is  the  maximizer.  Blue  receives  the 
negative  of  Red's  gain. 

CJiance 


U2P12+  v2q12 


B1 

B2 

B] 

R1 

uipn+  viqn 

"lpl2+  Vl1l2 

R1 

u2pn+v; 

R2 

ulp21+  v1^21 

ulp22+  vl‘>22 

*2 

u2p21+  v; 

For  reasons  which  are  intuitively  obvious  when  the  event  types  are 
related  to  the  strategic  choices,  we  will  order  the  probabilities  as 
follows : 


11 

qll 

P1 2 

>  q12 

21 

c 

q21 

P22 

q22 

11 

> 

P21 

qll 

*  q21 

12 

p22 

ql  2 

q22 
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To  fix  attention  on  a  representative  system  of  probabilities  compatible 
with  this  ordering,  consider  the  numerical  values; 


bi 

B2 

((pij  ■  v>  * 

R! 

(1/2, 1/2) 

(1,0) 

-  -  - 

-  -  —  -  -  -- 

* 

R2 

(1/4,  3/4) 

(1/2,  1/2) 

With  the  unit  weights  used  in  the  previous  paper  we  would  have  the  games 


G^  . 

G2 

B1  !B2 

B! 

<  B2 

Ri 

0  '  1 
i 

Ri 

1/2  i  1 

R2 

1 

-1/2,  0 
l 

R2 

1/4 

;  1/2 

The  utilities  are  generally  unknown  but  may  be  ordered  by 
analysis  of  the  mission  types.  In  fact,  we  will  assume,  for  the  present 
purposes,  that  u^  >  0  and  v^  <  0  (for  Red:  the  other  way  round  for 
Blue).  In  case  of  mission  type  ,  however,  we  will  reason  that  the 
event  type  "counterdetection"  is  usually  found  only  in  association  with 
"detection"  but  that  "detection"  may  also  occur  alone  at  longer  range. 
Accordingly,  while  both  have  negative  utility  for  Red,  in  the  light  of 
his  specific  mission  in  ,  we  will  assume  u~,  <  <  0  .  To  fix 


* 

Uj  =  u2  =  * »  =  -1 1  =  0  for  example;  unfortunately  the  actual 

Gj  and  G^  analyzed  in  the  previous  paper  cannot  be  obtained  from 
our  general  model  due  to  a  lack  of  adequate  attention  to  realism  in 
the  earlier  example. 
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on  a  definite  analysis  of  the  complete  I-game  model  we  will  examine 
the  case  where  Vj/uj  =  -2  and  v2/u2  ~  ^us  we  can  wr^e: 

U1  =  U1  '  V1  =  ”2ul 
u2  =  -4u'2i  v2  =  -3u'2 

with  u'j  and  u'.,  being  non-negative  unknown  constants.  Now  the 
I-game  model  must  be  represented  as: 


Chance 


R1 

(-1  /2)u'j 

:  “i 

«2 

(-5/4)u'j 

!  (-1  /2)u'j 

(-7/2)u' 


Rg  (-13/4)u'2 


(-7/2)u'2 


Solving  the  two  games  G,  and  G2  ,  we  observe  that  Gj  has  a  saddle- 
point  (Rj.Bj)  so  that  v(Gj)  =  -u'j/2;  while  G2  has  no  saddlepoint, 
but  the  standard  solution  gives  v(G2)  =  (-10/3)u'2  . 

j(C 

The'laverage"  game  G  (p)  is  obtained  if  the  two  sides  both 
receive  no  special  information  on  the  outcome  of  the  chance  move: 

G*(p) 


pu'j  +  (l-p)u'2  (-3) 


Rj  pu'j  (-1/2)+  (l-p)u‘2  (-7/2) 

R2  pu'j  (-5/4)+  (l-p)u'2  (-13/4) 


pu’j  (-1/2)  +  (l-p)u2  (-7/2) 


For  convenience  let  a  =  pu'j  and  b  =  (l-p)u'2 
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The  value  of  G  (p)  is  given  by  the  following  formula  unless  there  is 
a  saddlepoint: 


v*(p) 


(a  +  7b)2  -f  (a  -  3b)  (5a  +  13b)]  /4 
[  -a  -  7b  +  a/4  +  25b/4] 


3a2  +  6ab  4-  5b 
a  +  b 


j|( 

Since  the  matrix  of  G  as  a  function  of  a  and  b  is: 

-(a/2  +  7b/2)  a  -  3b 
-(5a/4  +  13b/4  -(a/2  +  7b/2) 

when  b  =  0,  there  is  a  saddlepoint  (Rj,Bj)  corresponding  to  the 

solution  of  Gj.  To  generalize  we  must  ask:  What  values  of  a  and  b 

are  feasible  and  satisfy  -(5a/4  +  13b/4)  <  -  (a/2  +  7b/2)  <  a  -  3b  ? 

The  other  entries  in  the  matrix  clearly  can  never  (i.  e.  ■,  for  no  a,  b  •>  0) 

be  8addlepoints,  The  necessary  and  sufficient  condition  is:  b  3a. 

$ 

Therefore,  the  complete  solution  for  G  (p)  is  as  follows: 


v*(p)  = 


-2(3a2  +  6ab  +  5b2)/3(a+b)  if  b  >  3a 


-(a/2  +  7b/2) 


if  b  ^  3a. 


As  shown  in  the  previous  paper,  the  I-game  need  not  be 
adequately  represented  by  the  average  game  G*(p).  Red  may  take 
advantage  of  the  fact  that  he  knows  his  mission  type:  the  so-called 
chance  move  at  the  top  of  the  diagrams  is  ,  of  course ,  a  fiction  which 
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is  designed  to  take  account  of  Blue's  willingness  to  choose  his  strategies 
by  Bayesian  techniques  with  the  prior  probabilities  (p,  1-p).  Explora¬ 
tion  of  Red's  potential  advantages  requires  that  we  analyze  the  four 

strategy  choices  R  ,  R  ,  R0.  ,  R__  where  :  R..  =  "choose  R.  if  G.  , 

11  lc  cl  dd  i  1 

and  R.  if  G_  is  the  'true'  game  The  game  matrix  for  this  4x2 
strategic  conflict  model  together  with  the  six  2x2  subgame  matrices 
are  given  in  Table  1  below.  All  utilities  have  been  multiplied  by  -J  in 
this  table . 


Table  1 


piy  (1/2)  +  (1-p)  u2'(7/2)  -pvij'  +  3  (1-p)  u2* 

puj'  (1/2)  +  (1-p)  u2'(13/4)  |  -puj'  +  (1-p)  u2'  (7/2) 

PUj*  (5/4)  +  (1-p)  u2'(7/2)  pu^  (7/2)  +  3  (1-p)  u2' 

pUj'  (5/4)  +  (1-p)  u2’(13/4)  '  pUl'  (1/2)  +  (1-p)  u2'(7/2) 


/2  +  7b/2  '.-a  +  3b 


I 

/2+13b/4  '-a  +  7b/2 

* 

t 


/  2  +  7b/2  ',-a  +  3b 


Formula  for  Value  of  Subgame 
When  There  is  No  Saddlenoint 


(a+7b)  (-2a+7b)  -  (2a+13b)  (-a+3b) 


(0.  a  +  3b/ 4)  4 


""  —  — "  1  *  ~ 

5a/ 4  +  7b/2  '.7a/2  +  3b 

1 

21 

1 5a  +  0 .  b 

a/2  +  7b/2  I- a  +  3b 
< 

11 

(a+7b)2  +  (a- 3b) (5a+13b) 

5a/4+  13b/4-  a/2  +  7b/2 

I 

i/2  +  13b/ 4  !-a  +  7b/2 


(2a-f  13b)  (a+7b)  +  (2a- 7b)  (5a+13b) 


/2  +  13b/ 4 


5a/  4  +  7b/2 

7a/2  +  3b 

r 

5a/ 4  +  7b/2 

7 a/2  +  3b  2l  _  (5a+i4b)  (a+7b)  -  (5a+13b)  (7a+6b) 

5a/ 4  +  13b/ 4  J  a/2  +  7b/2 


-  -  -  r'V»V  ^  *  ',‘i!1  -  '  ,'  *  A  ’  >"  '• 

\  A'S'T'  ,%"■%";>  »  •  *-%*  *-  /  1  V 


[ 

[ 

[ 

[ 

[ 

[ 

L 

L 

L 

L 

L 

L 


All  six  of  the  possible  2x2  subgames  must  be  solved  in  order  to  find 
the  correct  solution  of  the  4x2  game.  The  minimum  of  their  six 
separate  values  is  the  value  of  the  game  G(p)  .  Of  course,  the 
minimum  may  shift  about  as  p,  Uj’  and  u^'  are  varied,  so  that 
we  must  also  take  into  account  the  conditions  under  which  the  minimum 
value  is  attained  at  a  particular  subgame.  We  shall  express  these 
conditions  in  terms  of  a  and  b  for  the  present  purposes.  Table  2 


gives  the  possible  saddlepoints.  The  subgames  are  listed  as  G. 
according  to  the  pair  of  Red  strategies  R-  selected. 


kl 

ij 
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*****  ZuMfe*  A,  A 


Table  2 


Subgame  Conditions  for  Saddlepoint 

g\\  -a  +  7b/2  <  a/2  +  13b/4  <  a/2  +  ?b/2 

.  if  b  <  6a 

-a  +  3b  <  a/2  +  7b/2  <  5a/4  +  7b/2 

for  all  a,  b  >  0 

G22  -a  +  3b  <  a/2  f  7b/2  <  5a/4+  13b/4 

if  b<  3a 

GJ2  -a  +  7b/2  <  a/2  4  13b/ 4  <  5a/4+13b/4 

if  b  <  6a 

gJi  -a  +  7b/2  <  a/2+13b/4<  5a/4+7b/2 

if  b  <  6a 

G22  a/2  +  7b/2  <  5a/4+13b/4<  5a/4  +  7b/2 

if  b  <  6a 
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Saddlepoint 
(Rj2»  Bi) 

(Rjx'.Bj) 

(Rjj.Bj) 

(Rj2»Bj) 

(Rj2»Bj) 

(R22»Bj) 


Now  we  can  compare  the  six  values  of  the  subgames  and  seek  out 
the  maximum. 


Table  3 


Subgame 


Value 


rll 

G12 

-a/2  -  13b/ 4 
- 1  Ob/ 3 

b  <  6a 

b  >  6a 

pl 1 
°21 

-a/2  -  7b/2 

all  a,  b  >  0 

rn 

G22 

-a/2  -  7b/2 

-[2  (a+b)  +  4bZ/3  (a+b)] 

b  <  3a 

b  >  3a 

r12 

G22 

-a/2  -  13b/ 4 

all  a,  h>0 

C12 

G21 

-a/2  -  13b/ 4 
-  3  [(a+5b)  +  aZ/(5a-b)] 

b  <  6a 

b  >  6a 

r21 

G22 

-  5a/ 4  -  13b/ 4 
-  |  [(2a+5b)  +  aZ/(4a-b)] 

b  <  6a 

b  >  6a 
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The  maximum  of  the  six  values  is  easily  found  by  examination 
of  the  signs  of  all  possible  differences,  and  gives  the  following  function 
of  a  and  b  for  the  value  of  G(p): 


-a/2  -  13b /4  if  0  <  b  <6a  (G1*  or  G1*) 

v(p)  =  ,, 

-10b/3  if  6a  <  b  (G“) 

,  iff 

It  will  be  seen  now  that  v(p)*>v  (p)  for  all  a  and  b,  and  therefore 
this  holds  for  all  p  regardless  of  the  values  of  the  utilities:  u^1  and 
The  decision  as  to  which  game,  G(p)  or  (p)  to  play  belongs 
to  Red  who  possesses  the  advantage  in  knowing  which  is  the  true  game 
(Gj  or  G2).  As  the  maximizer,  he  naturally  prefers  G(p).  Blue 
can  make  the  same  calculations  and  so  he  can  and  should  hold  Red  to 
the  value  of  the  game  G(p);  any  other  Blue  strategy  results  in  a  worse 
outcome  for  Blue.  Optimal  strategies  in  terms  of  parameters  a  and 
b  are: 


(i)  if  o  <  b  £  6a 

(ii)  if  b  >  6a 


if  true  game  is  Gj,  . 
if  true  game  is  G2 


Note  that  Red’s  mixed  strategy  in  case  (ii)  refers  to  the  pure 
strategies  R^  and  R^»  not  to  R^  and  R2.  The  mixed  strategy  can  be 
interpreted:  if  G^  is  the  true  game,  choose  R^  (with  probability  one), 

but  if  G2  is  the  true  game,  choose/l  wlth  ProbablUt'r  ^ 

I  ^2  probability  6a  -f  2b 
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CHAPTER  VIII 

FORMULATION  OF  A  GAME  OF 
SEARCH  AND  PURSUIT 


John  P.  Mayberry 
Francis  M.  Sand 
Alan  J.  Truelove 


FORMULATION  OF  A  GAME  OF  SEARCH  AND  PURSUIT 

/ 

I.  INTRODUCTION 

In  this  paper  we  formulate  a  two-stage  game,  whose  first  stage 
is  a  search  game  (in  which  very  little  information  is  available  to  the 
antagonists  ),  and  whose  second  stage  is  a  pursuit  game  (which  is  a 
differential  game  of  perfect  information).  The  combined  game  is 
intended  to  represent  aspects  of  realistic  ASW  situations ,  where  a 
search  may befollowed  (in  case  the  searcher  is  successful  in  locating 
the  evader)  by  a  pursuit;  or  where  a  submarine,  with  limited  range  and 

endurance  while  submerged,  must  try  to  progress  towards  his  objective 

/ 

without  making  his  surfacing  point  too  predictable . 

Of  course,  the  behavior  of  both  the  pursuer  P  and  the  evader  E, 
during  the  first  stage ,  will  be  influenced  by  their  knowledge  that  the 
second  stage  will  be  played  at  some  future  time .  The  nature  and  extent 
of  that  influence  may  be  little  or  great,  depending  on  the  parameters  of 
the  game  (principally  the  ratio  w  of  E's  speed  to  P'e  speed,  and  the 
radius  r  of  P's  capture -circle),  on  the  initial  conditions  (principally 
the  initial  positions  of  E  and  P,  relative  toE's  destination),  or  the 
nature  of  E's  destination  (which  we  assume,  throughout  this  paper,  t" 
be  a  straight  line  called  the  "life-line"),  and  on  the  "re-detection 
time,"  when  stage  1  ends  and  stage  2  begins  (which  may  be  random,  or 
may  Occur  when  one  or  the  other  of  P  or  E  crosses  some  given  curve). 
Several  interesting  variants  will  be  suggested  below. 


In  section  2  below,  we  formulate  the  two-stage  game  more 
precisely;  in  section  3  we  present  in  some  detail  the  solution  to  the 
second  stage  (given  the  initial  conditions  to  that  stage),  for  the  special 
ca«e  whe/e  the  capture-zone  is  negligibly  small,  so  that  P  must  contact 
E  to  capture;  in  Section  4,  we  describe  how  we  intend  to  proceed  in  order 
to  solve  the  first  stage  (and  thus  to  solve  the  whole  game);  and  in  section 
5,  we  describe  how  the  results  of  these  two-stage  games  could  be  of 
practical  value  in  certain  ASW  situations  . 

2.  FORMULATION 

We  may  guide  our  intuition  by  thinking  of  a  submarine  as  the 
evader  E ,  a  destroyer  as  the  pursuer  P,  and  a  sub-pen  area  as  the  life¬ 
line  A.  At  time  t  =  0,  when  P  and  E  each  discover  the  other's 
l,oc  at  i  o  n  ,  the  sub  dives  and  both  proceed  quietly  (neither  detectable 
by  the  other)  until  redetection  occurs .  At  that  time  ,  *+»<*e  2  begins;  both 
proceed  at  top  speed  (in  full  awareness  of  each  othe  »  •  <•  *tion)  to  see 
whether  or  not  P  can  capture  E  before  E  reaches  l  . 

This  explanation  implies  that  the  absolute  speeds  of  bothP  and 
E  are  greater  in  stage  2 ,  but  it  is  only  the  ratio  of  the  speeds  which 
enters  into  the  mathematics  of  the  problem;  we  simplify  matters  by 
assuming  that  the  ratio  w  of  E's  speed  to  Ps  speed  does  not  change  when 
we  go  from  stage  1  to  stage  2 . 

We  quote  from  the  description  of  the  lifeline  game  in  [DG] ,  changing 
to  our  own  notation: 

''The  two  points  P  and  E  have  each  simple  motio  i*  in  a  half- 
plane  bounded  by  a  line  K  Their  speeds  are  parameters  and  as  only 
the  ratio  matters ,  we  shall  let  P  have  speed  unity  and  E,  speed  w(w  =  1). 

* 

i.e. ,  each  moves  at  a  fixed,  preassigned  speed,  and  each  has  complete 
control  over  his  own  direction  of  motion  at  any  time  . 
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Capture .  .  .is  to  occur  when  the  distance  PE  <  r  . . .  E's  objective  is  to 
reach  &  prior  to  capture  and,  naturally,  P's  objective  is  to  prevent  him  . " 

To  complete  the  description  of  our  two-stage  game,  we  need 
only  add  these  conditions: 

At  t  =  0 ,  they  know  each  other's  locations;  for  0  <  t  <  t^ ,  neither 
gains  more  information;  at  t  =  t^  ,  stage  1  is  over;  for  t  >  t^ ,  stage  2  is 
played,  starting  from  the  locations  of  the  participants  at  t  =  tj  ,  with 
perfect  information.  Of  course,  the  final  outcome  of  stage  2  (viz. 
whether  E  escapes  or  is  captured)  will  be  completely  determined  by 
the  players'  locations  at  t  =  t  ,  in  a  way  which  will  be  made  explicit 
in  section  3  below . 

Now  let  us  present  some  preliminary  remarks: 

(i)  If  w  >  1 ,  the  evader  can  always  escape  in  stage  2  --  unless 
capture  occurs  at  t  =  tj  .  Because  of  this  exception,  we  did  not  wish 
to  ignore  the  case  w  >  1 . 

(ii)  If  the  initial  locations  at  t  =  0  would  have  permitted  E  to  escape 
when  P  could  see  and  pursue  him,  then  the  same  action  by  E  will  result  in 
escape  when  P  cannot  see  him. 

(iii)  If  w  >  1 ,  then  E  could  escape  without  stage  1;  then,  from  (ii) 
above,  we  see  that  E  can  escape  anyway. 

(iv)  If  tj  is  small,  then  the  initial  interval  represented  by  stage  1 
of  the  game  will  have  little  influence  on  the  outcome  . 

(v)  If  ^  is  sufficiently  large ,  E  can  reach  &  before  t^  and  will 

escape . 

(vi)  When  tj  =  0  ,  this  2-stage  game  becomes  ths  lifeline  game; 
ref  [DG]  and  section  3  describe  how  to  determine  the  capture  zone  CZ 
and  escape  zone  EZ. 
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(vii)  In  4  jneral,  optimal  play  in  stage  1  will  involve  mixed 
strategies ,  so  we  Mil  be  forced  to  compute  probability  of  escape 
rather  than  EZ  and  CZ  .  Probability  of  escape  will  depend  on  initial 
locations  P  and  E,  on  the  speed-ratio  w,  on  the  capture -radius  r,  and 
on  the  first- stage  duration  t.  . 

(viii)  For  t,  > -  ,  P  =  1 . 

1  w  e 

(ix)  If  P  and  E  are  such  that  E  is  in  EZ  (P,  w ,  r ),  then  P  (P,E  ,w ,r  ,t. )  =  1 

e  1 

(x)  P  (P,E  ,w ,r  ,t, )  <  P  (P,E  ,w  ,r  ,t, ')  if  t,  <  t, in  other  words  , 
the  longer  the  "blind- search”  period,  the  better  E*s  chances  of  escaping. 

(xi)  If  w  =  1 ,  a  special  problem  arises;  the  second  stage  is  then 
a  simpler,  but  different,  problem. 

3 .  ANALYSIS  OF  THE  SECOND  STAGE  AS  A  LIFE-LINE  GAME 

E  is  trying  to  reach  the  destination  line  P  is  trying  to  intercept 
E  at  or  before  the  time  E  reaches  tlr  barrier . 

At  time  t  =  t^,  let  the  location  of  the  two  players  with  respect  to  £ 
be  shown  in  Figure  1 .  E  will  be  taken  aB  the  origin  of  coordinates  (0,0); 

E  and  the  line  P  (y  =  -d)  will,  for  the  moment,  be  regarded  as  fixed,  and 
the  position  of  P  will  be  denoted  by  (p  cos  d  ,  p  sin  d),  assumed  variable . 

At  all  later  times  t  >  t^  ,  both  P  and  E  know  each  other's  position. 


Recall  that 


/e's  speed \ 
l  P's  speed  / 


(which  we  now  will  assume  <  1  because  of  remarks  (iii),  (xi),  above). 

We  present  explicit  solutions  in  the  case  r  =  0,  when  P  must  Actually 
meet  E  in  order  to  effect  capture.  As  reference  [DG]  shows,  the  case  r>  0 
is  qualtitatively  similar  in  many  respects  ,  but  the  explicit  solutions  would 
certainly  be  much  more  complicated. 
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(a)  Case  r=0;  The  circle  of  Appolonius 


(b)  Case  r  >  0:  A  curve  of  the  Fourth  Degree 
(Biquadratic) 


Figure  1 

Boundary  curves  between  capture  and  escape  (Stage  2) 
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The  locus  C  of  points  X,  such  that  E  and  P  could  reach  X  in  the 
same  length  of  time, is  a  circle,  the  circle  of  Appolonius;  see  Figure  1(a). 

In  the  case  r  >  0,  the  circle  is  replaced  by  a  biquadratic  curve  C  of  similar 
appearance;  see  Figure  1(b).  The  points  inside  C  (including  E  itself)  are 
points  which  E  could  reach  sooner  than  P  could;  the  points  outside  C 
(including  P  itself)  are  points  which  P  could  reach  sooner  than  E  could. 

A  diameter  of  C  is  identified  by  the  two  points  A  ^  and  A^  on  the  line  PE 
which  satisfy 

lA.Et  =  |A2E|  .  w  _ 

lVl  lApl 

If  C  does  not  meet  £  ,  then  P  can  block  E  by  heading  for  the  point 
of  £  closest  to  C;  P  would  reach  that  point  before  E  could.  Of  course  , 
as  E  and  P  move ,  the  circle  C  may  also  move;  but  C  cannot  move  so  as 
to  meet  £  unless  P  makes  a  mistake  ,*  and  as  long  as  C  does  not  meet  £  , 

P  can  remain  "below”  E,  moving  upward  to  capture  at  some  later  time. 

If  C  meets  £  at  two  points  ,  then  E  can  escape  from  P  by  heading 
for  a  point  of  £  between  them;  E  will  arrive  there  first,  and  escape . 

(See  remark  (ii)  above  . ) 

Considering  E  fixed  and  P  variable  ,  the  locus  of  points  P,  such  that 
P  would  just  barely  capture  E,  is  the  locus  of  points  P,  such  that  the  circle 
of  Appolonius  C  is  tangent  to  £  ,  as  shown  in  Figure  2 .  That  locus  consists 
of  the  boundary  of  a  half-ellipse  ,  as  shown  in  Figure  3.  We  call  the  interior 
of  that  half-ellipse  the  capture-area  CA ,  since  it  consists  of  possible 
locations  for  P  where  capture  would  certainly  result  if  both  antagonists 
behaved  optimally;  its  exterior,  consisting  of  possible  locations  for  P  where 
escape  would  certainly  result  if  both  antagonists  behaved  optimally,  is 
called  the  escape  area  EA  .  Note  that  the  CA  and  the  EA  are  iii  the  space 
of  locations  for  P. 


///  /  7~T7~7~7~7  7/7/  /  / /  /  /  / 7  7  7 

Figure  3 

The  capture-area  CA  and  the  Escape-area  EA  in  the  space  of 
locations  for  P. 


implies  capture  ; 
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Now  consider  P  fixed,  and  E  variable.  Now  the  set  of  points 
where  E  could  bv  located,  and  still  escape,  is  the  outside  of  one  limb 
of  a  hyperbola,  as  shown  in  Figure  4  .  That  region  is  the  escape  zone, 
EZ.  The  "inside*1  of  that  hyperbola,  including,  of  course  ,  all  points 
directly  above  P,  is  the  set  of  points  where  E  could  be  captured;  it  is 
the  capture-zone  CZ.  Note  that  the  EZ  and  the  CZ  are  in  the  space  of 
locations  for  E . 

The  condition  for  eventual  capture  ,  assuming  optimal  play  in 
stage  2 ,  is  that 


at  the  beginning  of  stage  2 ,  where  d  is  the  distance  of  E  from  l  ,  y  is  the 
distance  of  P  from  K>  ,  and  x  is  the  "horizontal"  component*  of  the 
separation  between  E  and  P. 

We  define  the  function  capt(x,y,d)  by  the  relation: 


capt  (x.y.d) 


,  2  2  .  2  f  >0  , 

2  k ,  y  ,  ,  ,x  ,  1  -w  /  7 

)(■£)  +  (t  )  5 r  < 


if  (1-w  )($>  +  §) 


w 


1 


Thus  capt  (x,y,d)  =  1  if, and  only  if,  capture  can  be  forced  by  P  in  stage  2 . 


*(i.e  .,  parallel  to  l  ) 
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4. 


APPROACH  TO  SOLUTION  OF  STAGE  I 


Using  the  results  of  section  3  above,  we  can  predict  with  certainty 
whether  or  not  a  given  system  configuration  (i.  e. ,  the  locations  of  E 
and  P  relative  to  t  ),  at  the  beginning  of  stage  2,  would  result  in 
capture  of  E  by  P  during  stage  2,  assuming  optimal  action  by  both 
in  sate  2.  The  formulae  for  malting  that  prediction  were  supplied  ex¬ 
plicitly  in  section  3  for  the  case  r-0  ;  the  case  r  >  0  is  qualitatively 
similar  (although  it  favors  P  ,  and  is  therefore  quantitatively  not 
correct). 

Let  us  now  consider  stage  1  as  a  game,  whose  payoffs  are 
determined  by  the  locations  of  P  and  E  at  the  end  of  state  1 .  W~ 
shall  first  consider  the  game  of  kind,  where  the  payoffs  are  the  logical 
alternatives:  "E  wins",  or  "P  wins".  We  will  identify  P  as  the  max¬ 
imizing  player,  and  say  that  the  "payoff"  (meaning,  the  payoff  from  E 
to  P  )  is  =  1  if  capture  can  be  guaranteed  from  the  system  configuration 
at  the  end  of  state  1,  and  is  =  0  if  escape  can  be  guaranteed  from  the 
system  configuration  at  the  end  of  stage  1.  (The  analysis  of  stage  2  as 
a  pursuit  game  shows  that  one  or  other  of  these  conditions  must  hold.  ) 

Then  P's  goal  is  to  maximize  the  expected  value  of  this 
numerical  payoff  — or,  what  is  the  same  thing,  the  probability  of 
capture —  while  E's  goal  is  exactly  opposite. 

The  strategies  which  are  open  to  each  of  P  and  E  comprise 
a  whole  function  space  of  paths  which  each  might  follow,  during  the 
time  0<  t  <tj,  of  stage  1.  However,  it  is  obvious  that  the  path  followed 
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by  either  is  totally  irrelevant;  the  only  things  that  matter  are  the  end- 
points--their  locations  at  time  t  =  t^  when  re-detection  occurs .  Thus 
we  can  identify  the  pure  strategies  for  each  player  with  the  set  of  points 
at  which  they  could  be  located  at  time  t ^ ;  in  each  case  ,  this  will  be  a 
circular  disk,  centered  at  his  original  (t  =  0)  position  (Eq  or  P  ,  as  the 
case  may  be),  omitting  any  part  of  the  disk  which  might  be  on  the  wrong 
(inaccessible)  side  of  Recall  that  P's  velocity  was  taken  as  1,  and 
E's  asw;  thus,  the  pure  strategies  of  P  correspond  to  a  circular  disk 
o®pcentered  at  Pq  and  with  radius  t ^  ,  while  the  pure  strategies  of  E  '  x 

correspond  to  a  ci--cular  diskc^,  centered  at  E  and  with  radius  wt  , 
each  restricted  to  the  accessible  side  of  (see  Figure  5). 

Now,  although  all  those  strategies  are  legal,  not  all  are  plausible; 
simple  arguments  will  further  restrict  the  set  of  rational  strategies  for 
each  player  .  In  particular  ,  note,from  Figures3  and  4 ,  that  it  will  never 
profit  either  player,  no  matter  where  the  other  is,  to  be  farther  from 
and  opposite  the  same  point  of  .  It  follows  that,  in  determining  what 
constitutes  rational  play  in  this  game  of  kind,  we  need  only  consider  the 
points  of  those  two  cir  disks  which  are  not  separated  from-^  by 

other  points  of  the  sa.i  _sks--i,e.  ,  the  semi-circles  bounding  the  disks 
on  the  side  towards  .  (If  a  disk  intersected-^  ,  then  the  set  of  "nearest 
points"  would  possess  a  linear  portion  between  two  circular  arcs.)  Those 
semi-circles,  whose  points  now  represent  plausible  strategies,  are 
labeled^  and  ,a£on  Figure  6.  The  points  X  of  those  semi-circles,  and  thus 
the  strategies,  can  be  identified  by  the  angle  between  XPq  (or  XEq;  and  -C-; 
a  pure  strategy  for  P  is  denoted  y  ct>  and  a  pure  strategy  for  E  by  the 
resulting  positions  of  P  and  E  ictively ,  at  time  t  =  t^  ,  will  be  denoted 


by  Pj  (a)  and  Ej  03). 
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Figure  5 

Feasible  strategies  in  Stage  1 


Figure  6 

Plausible  Strategies  in  Stage 1. 


i 


VIII- 14  ! 

1 


To  recapitulate:  this  stage  1  game,  denoted  by  I",  has  players  P 
and  E;  P  has  pure  strategies  a,  0  <  a<  ft  5  E  has  pure  strategies  /),  0  < 
fi<Tt  l  and  the  payoff  p  (a,  /3 )  is  defined  by 
p  (a,  jS)  =  capt  (x,  y,  d), 

where 


x  =  x-coord.  of  Pj  (cf)  -  x-coord.  of  ^ 

y  =  y-coord.  of  P^  (od  , 

d  =  y-coord.  of  E^  (0)  •  J  A 

The  we  must  ask,  "Is  this  gameP strictly-detfcrmiri®?"  --  i.e. , 
"Is  there  a  way  P  can  be  sure  of  capturing,  pr  a  way  E  can  be  sure  of 
escaping?"  First  of  all,  ifjsP  meets -^then  E  can  always  escape  before 
stage  2  starts.  Secondly,  if  Eq,  P  ,  w  and  r  are  such  that  E  could 
escape  in  a  (stage -2)  pursuit  game  whose  starting  positions  were  Eq  and 
P^  ,  then  E  could  in  P follow  the  same  path,  P  could  accomplish  no 
more  in  P (where  he  has  less  information),  and  E  will  surely  escape  in 
FI  So  much  is  obvious;  but  the  converse  is  also  true:  if  E  has  a  course 
of  action  inF  which  wiil  surely  result  in  escape  ,  but  which  does  not  allow 
F  ;o  reach-^before  t^  ,  then  that  course  of  action  would  result  in  escape 
even  if  P  knew  of  it,  and  E  would,  therefore ,  escape  in  the  pursuit  game 
(i.e .  ,  even  if  t^  had  been  =  0). 

At  the  opposite  extreme  ,  P  might  be  able  to  force  capture  .  That 

will  be  possible  if  there  is  some  r>o’nt  P°  gjQ^Lwhose  capture-zone  CZ 

0  *  o 

includes  the  whole  of.#  .  The  existence  of  such  a  point  P  --or  its 

E 

location,  if  one  does  exist  --  cannot  always  be  guessed.  See  Figure  7 
for  an  example  . 

In  each  of  these  cases  --  where  the  probability  of  escape  is  either 
0  or  i  --  it  is  reasonable  to  consider  the  associated  game  of  degree , 
where  a  new  payoff  is  introduced, depending  on  the  length  of  time  before 
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capture  (or  before  E  reaches  ,  as  the  case  may  be).  In  this  way 
we  can  remove  the  ambiguity  in  the  action  recommendations  which 
often  characterizes  a  differential  game  of  kind.  We  must  be  careful, 
in  considering  the  game  of  degreeP'  associated  with  a  strictly-determined 
game  of  kindP,  not  to  allow  the  players  inP’  all  the  strategies  of  P; 
the  player  who  can  winP will,  in  general,  be  forced  to  avoid  certain 
of  his  strategies  inP,  and  he  must  avoid  those  strategies  inP'  also. 

In  the  example  of  Figure  7,  P  is  restricted  (if  he  is  to  win  the  game  of 
kind)  to  strategies  very  near  to  P°;  if  he  wished  to  minimize  time  to 
capture,  he  might  consider  a  direct  chase  (from  Pq  towards  Ec),  but 
would  then  find  himself  near  P^  at  time  t^ ,  and  E  would  escape . 

Solution  of  the  game  of  kind  ,  in  cases  where  neither  E  nor  P  can 
force  a  win,  requires  consideration  of  mixed  strategies  on  each  side. 

Simple  geometric  arguments  will  probably  not  suffice  to  determine  the 
nature  of  these  mixed -strategy  solutions. 

5.  APPLICATION  OF  THIS  TWO-STAGE  GAME  TO  REAL  PROBLEMS 
OF  ASW 

This  game ,  as  described  in  section  2 ,  is  not  very  close*  +p  any 
realistic  ASW  situation.  Nevertheless ,  there  are  a  number  of  directions 
in  which  modifications  could  be  made  which  would  result  in  valuable  in¬ 
sights  towards  actual  ASW  problems  . 

(a)  We  might  consider  a  situation  where  t^  was  not  known  'ith 
certainty,  but  where  a  probability-distribution  of  redetection  times  was 
known  to  both  P  and  E . 

(b)  We  might  consider  that,  at  time  tj ,  one  or  both  of  P  and  E  rn^ht 
be  uncertain  of  redetecting  the  other  -  -  i.e  . ,  each  would  have  only  a 
probability  of  detecting  the  other  . 
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(c)  Those  redetection  probabilities  might  depend  on  the  range 
(distance  between  and  E^). 

(d)  We  might  have  repetitive  intervals  of  undetectability,  with 
mutual  detection  between. 

(e)  Using  the  general  notion  of  a  game  of  incomplete  information 
(I-game),  as  in  Chapters  VI  and  VII  of  this  report,  we  may  include  mutual 
uncertainty  about  speeds,  objectives  (shape  and  location  of  Destination  - 
set  t),  and  detection-capabilities  . 


Reference: 

[DG]:  "Differential  Games',' ,  by  Rufus  Isaacs;  J.  Wiley  &  Sons  (1965) 
pp. 257-260. 


VIII- 18 


